Combined emoji are not supported in text fields

[grantfitzsimmons]

(1366, "Incorrect string value: '\\xF0\\x9F\\x91\\xA9\\xF0\\x9F...' for column `herb_rbge`.`collectingevent`.`Text1` at row 1")

Recreated in edge and testability.

Specify 7 Crash Report - 2022-11-25T19_04_56.813Z.txt

https://herbrbge-edge.test.specifysystems.org/specify/view/collectionobject/349648/?recordsetid=178 https://herbrbge-testability.test.specifysystems.org/specify/view/collectionobject/349648/?recordsetid=178

[maxpatiiuk]

:sadface: 😢

[grantfitzsimmons]

This is only for combined emoji like 👴🏻 (actually 👴 🏻) Regular emoji like ⚙️ are fine!

[grantfitzsimmons]

MySQL's utf8 only supports basic multilingual plane, and you need to use utf8mb4 instead:

For a supplementary character, utf8 cannot store the character at all, while utf8mb4 requires four bytes to store it. Since utf8 cannot store the character at all, you do not have any supplementary characters in utf8 columns and you need not worry about converting characters or losing data when upgrading utf8 data from older versions of MySQL.

So to support these characters, your MySQL needs to be 5.5+ and you need to use utf8mb4 everywhere. Connection encoding needs to be utf8mb4, character set needs to be utf8mb4 and collaction needs to be utf8mb4. For java it's still just "utf-8", but MySQL needs a distinction.

I don't know what driver you are using but a driver agnostic way to set connection charset is to send the query:

SET NAMES 'utf8mb4' Right after making the connection.

Source

[grantfitzsimmons]

From @carlosmbe:

Describe the bug If an Emoji is present in an Agent's First, Last Name or Middle Initial. Specify will crash

Agent Email is fair game though

To Reproduce Steps to reproduce the behavior:

1. Go to Data Entry -> Agents
2. Click on any of the fields stated above
3. Add Emoji and Save
4. See error Specify 7 Crash Report - 2023-04-27T21_26_02.395Z (1).txt Specify 7 Crash Report - 2023-04-27T21_26_02.395Z.txt

Expected behavior Not a crash or at least a request to remove UTF-8 Characters and only have ASII characters

Screenshots

Desktop: OS: Mac OSx Ventura Browser: Chrome Specify 7 Version: 7.8.10-prerelease

Database Name: morpaleo

Reported By @carlosmbe

[carlosmbe]

Edit: Still present in 7.8.10 pre release

Below is an instance that happened in Paleo Context Remarks

Specify 7 Crash Report - 2023-05-01T21_19_03.235Z.txt Specify 7 Crash Report - 2023-05-01T21_19_03.235Z (1).txt

https://user-images.githubusercontent.com/53784701/235533632-aa8926fa-13cf-4f10-b8c9-045b9d0da879.mov

[grantfitzsimmons]

Happens with Good Ol Fashioned Simple Emojis too. Below is an instance that happened in Paleo Context Remarks

Emojipedia: (Source)

The Flag: United States emoji is a flag sequence combining 🇺 Regional Indicator Symbol Letter U and 🇸 Regional Indicator Symbol Letter S. These display as a single emoji on supported platforms.

It is 🇺 + 🇸 = 🇺🇸

[emenslin]

Can recreate in edge (7.9.6)