Not all emojis can be represented as fixed 32 bit values

[jurriaan]

Noticed that in Waypoint and the Data messages emojis are represented using fixed 32 bit values:

https://github.com/meshtastic/protobufs/blob/dea3a82ef2accd25112b4ef1c6f8991b579740f4/meshtastic/mesh.proto#L778-L781 and https://github.com/meshtastic/protobufs/blob/dea3a82ef2accd25112b4ef1c6f8991b579740f4/meshtastic/mesh.proto#L731-L735

Noticed in the Android app that for waypoints some emojis don't work, or show different icons. For example, the pirate flag emoji just shows as a black flag on the map.

[garthvh]

Fixed32 works fine, must be an encoding / emoji issue on android

[jurriaan]

@garthvh that's a text message, not a waypoint.

If you look at the payload of that message (f09f8fb4e2808de298a0efb88f), you'll see that emojis like this pirate flag actually consists of multiple unicode codepoints. (The Pirate Flag emoji is a ZWJ sequence combining 🏴 Black Flag, Zero Width Joiner and ☠️ Skull and Crossbones. See https://emojipedia.org/pirate-flag)

If you create a waypoint using that emoji it won't work properly as it consists of multiple unicode codepoints, since the android app for example only uses the first codepoint, the black flag (🏴): https://github.com/meshtastic/Meshtastic-Android/blob/61be6e9985a0609423b62ff77ad7c2bc6684d61b/app/src/main/java/com/geeksville/mesh/ui/map/components/EditWaypointDialog.kt#L150

Which makes sense as you won't be able to fit these three characters in a 32 bit value.

[ianmcorvidae]

I'm not sure how desirable it is to support zero-width joiner emoji for waypoint icons like this; in theory those values could get pretty long (even just in the "recommended sequences" list the Unicode consortium puts out, some are as long as 8 code points, from what I found, and I would not be surprised if there are longer ones supported some places). The example pirate flag is already 4, when correctly encoded, since it needs the zero-width joiner and a variation selector. Maybe it's possible to leave some things like that off and compute a correct value, though I wouldn't necessarily count on it. No idea how much headroom we have in these messages right now/how much we'd want to allow.

The apps should probably at least be validating that waypoint icons are in fact within whatever limit is applicable (currently, one Unicode codepoint within the right ranges, presumably)

[jurriaan]

@ianmcorvidae Yes, I think that does make sense!

In that case it's probably good to update the protobuf documentation for those fields as well to include that supporting a single codepoint for emojis is a design decision.