Issues finding emoji data by native emoji entered via macOS keyboard?

[jessicalc]

Hello! Thanks for building emoji-mart, we (and our users) love it.

We're running into issues looking up emoji by their native characters. In our application, we look up emoji data by their native character in two ways:

1. Using the new async headless SearchIndex.search API, and
2. Waiting for Data.natives to load (after calling init), and getting Data.natives[nativeEmojiChar].

Unfortunately, we can’t use the async headless search in all parts of our application, which is why we sometimes have to use approach #2.

Most emoji are findable via their native character using both methods. However, I’ve encountered some emoji that aren’t findable via SearchIndex.search API, and others that aren’t findable in the Data.natives dictionary via their native character.

You can try using https://missiveapp.com/open/emoji-mart/example-headless-search.html to look up the ⭐️ emoji or the ☕️ emoji and easily repro the headless search problem.

To illustrate the Data.natives problem, I made a codesandbox here: https://codesandbox.io/s/trusting-cookies-zopvw4?file=/src/App.tsx

My question is:

Do you think there could be an issue with the source data in the native.json definition? Or is the macOS emoji input wrong? Or maybe it's a Chrome thing? Maybe it's a combo of all? (I don't know a whole lot about how this stuff works!)

Other probably helpful info:

• I used the emoji-unicode NPM package to debug and found some mismatches between the emoji in data.json and the emoji I entered using the macOS keyboard in Chrome, cross-referencing those against https://www.unicode.org/emoji/charts-14.0/full-emoji-list.html

Native emoji	emoji-mart's native.json	native Mac OS emoji unicode via emoji-unicode(<native emoji>) (npm)	Unicode chart
⭐️	2b50	2b50 fe0f	2b50
☕️	2615	2615 fe0f	2615
⚓️	2693	2693 fe0f	2693
🕷	1f577-fe0f	1f577	1f577
🌶	1f336-fe0f	1f336	1f336

I realize that some of the source of this problem may be related to how Apple/Chrome handle emoji inputs (esp for stuff like the star, coffee, anchor emoji)! But I would be grateful for your advice and am curious to know where or how you generate native.json.

Thank you so much for your attention!

[EtienneLem]

Thanks a lot for the thorough research and issue. This is something I did notice as well. Our data source is https://github.com/iamcal/emoji-data which itself seems to be based on Unicode. Ours seem to match the ones found on Emojipedia:

• https://emojipedia.org/star (⭐ U+2B50)
• https://emojipedia.org/spider (🕷 U+1F577,U+FE0F)

It feels like the outlier might be the macOS version? 🤔 But, whoever is wrong or right isn’t much relevant, we should find a way to support both versions. When building our own data, we can probably cross-reference with emoji-unicode and if the native emoji doesn’t match the one we have, we could add the other version to the search string and Data.natives. Our returned data when clicking an emoji would still use the version we’re currently using (the right one until proven otherwise 😆)

[jessicalc]

Here's the current copy of the native emoji overrides map that we've been maintaining on an ad-hoc basis: https://gist.github.com/jessicalc/efef219479afb0ecf5e65532809abf29