utf8mb4: keeping emoji and Uzbek letters intact in MySQL

A customer leaves a review on your site, adds a smiley at the end and clicks Submit. The review either fails to save, shows four question marks where the emoji was, or is cut off at exactly that point. This happens mostly on older or misconfigured MySQL databases, and the cause is almost always the same: the table or the connection is not using utf8mb4. Below we look at where the problem comes from, how it relates to the Uzbek alphabet and how to convert a database safely.

MySQL "utf8" is not full UTF-8

In UTF-8 a character takes 1 to 4 bytes. Latin letters and digits use 1 byte, Cyrillic (including ў, қ, ғ, ҳ) uses 2, many special symbols use 3, and emoji plus some rare characters need 4. Historically MySQL shipped a truncated variant under the name utf8 that only stores characters up to 3 bytes; current documentation calls it utf8mb3. Real UTF-8 in MySQL is called utf8mb4.

So when a table uses utf8, a 4-byte character simply does not fit. In strict mode the query fails; otherwise the character is replaced with a question mark or the value is truncated. The error log typically shows something like this:

SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xF0\x9F\x98\x80' for column 'comment' at row 1

Where Uzbek letters actually break

Uzbek o' and g' typed with a plain apostrophe are single-byte characters, the special ʻ sign (U+02BB) takes 3 bytes and Cyrillic ў and қ take 2, so all of them fit even in a utf8 (mb3) table. Uzbek text usually gets damaged for a different reason: the table was created as latin1 or cp1251, or PHP connects without specifying a charset. Cyrillic then gets stored as "????", and it cannot be recovered afterwards. Moving to utf8mb4 fixes both issues at once.

Checking the current state

SELECT TABLE_NAME, TABLE_COLLATION
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = 'example_db';

SHOW VARIABLES LIKE 'character_set%';

If you see latin1_swedish_ci or utf8_general_ci, the tables need converting. If they already show utf8mb4_unicode_ci or utf8mb4_0900_ai_ci, look at the connection settings instead.

Converting step by step

Always take a full backup first, and set the dump charset explicitly so the backup itself is not damaged:

mysqldump --default-character-set=utf8mb4 -u example_user -p example_db > example_db_backup.sql

Then convert the database and each table. CONVERT TO changes all text columns and re-encodes the existing data:

ALTER DATABASE example_db CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE comments CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

On old servers (MySQL 5.6 and earlier, especially with the COMPACT row format) you may hit "Specified key was too long; max key length is 767 bytes", because an index on VARCHAR(255) now needs 1020 bytes. MySQL 5.7+ with the DYNAMIC row format raises the limit to 3072 bytes; on older versions shorten the indexed column to VARCHAR(191).

Setting the charset in PHP

Converting tables is only half the job. If the application does not declare the charset when connecting, the server default is used and emoji break again:

$pdo = new PDO('mysql:host=localhost;dbname=example_db;charset=utf8mb4', $user, $pass);

$db = new mysqli('localhost', $user, $pass, 'example_db');
$db->set_charset('utf8mb4');

In WordPress the setting is define('DB_CHARSET', 'utf8mb4'); in wp-config.php. Laravel already defaults to utf8mb4 in config/database.php, but older projects are worth checking. Replace any legacy SET NAMES utf8 with SET NAMES utf8mb4.

Example: a flower shop in Tashkent

A flower delivery service in Tashkent runs misol.uz with an order form, and customers often add 🌹 or 🎉 to their greeting text. The orders table was created years ago in utf8_general_ci, so orders with emoji never reached the database and the operator never saw them. The developer took a backup, converted orders and order_items, added charset=utf8mb4 to the PDO DSN and placed a test order; greetings now save in full. Do this during low-traffic hours, since converting a large table can lock it for several minutes. On sayt.uz hosting you can do it in phpMyAdmin, opened from the sayt.uz cabinet.

After the conversion

Save a record containing emoji, Cyrillic ў and қ, and Latin o' and g', and check how it renders.
Make sure pages include <meta charset="utf-8">.
Test search and sorting, because collation changes comparison rules.
Text that was already stored as "????" cannot be fixed by changing the charset; only a restore from backup helps.

In short, utf8mb4 protects future data but does not bring back lost characters, so create new databases with utf8mb4_unicode_ci from day one.