chat: render emojis

[yarikoptic]

emojis already used for reactions etc. But they are absent where IMHO most needed -- in the chat. I wonder if it might be easy to implement using e.g. https://www.npmjs.com/package/react-emoji-render to render chat messages? I think it would be ok to not have some fancy completions etc to start with and just rely on people to know to type :smile: or :)

---

Edit by @0xdevalias: See also https://github.com/sparkletown/sparkle/issues/1395 as a tightly related issue/feature.

[0xdevalias]

@yarikoptic Interesting you should raise this, as it's something I had looked into a bit and included on part of our backlog. I hadn't gone deep on making all of the decisions/explorations around this yet, but I'll include relevant snippets below (note: you won't be able to see any of the 'internal' issues that are linked)

See also my notes in https://github.com/sparkletown/sparkle/issues/1395#issuecomment-841594823 that refer to https://github.com/sparkletown/internal-sparkle-issues/issues/188#issuecomment-817416820

---

4. Emoji support

Originally posted by @jess-sparkles in https://github.com/sparkletown/internal-sparkle-issues/issues/484#issuecomment-796006117

---

Some resources I stumbled across the other day, capturing here for posterity:

• https://github.com/missive/emoji-mart

  • Emoji Mart is a Slack-like customizable emoji picker component for React

  • https://github.com/missive/emoji-mart#custom-emojis

  • You can provide custom emojis which will show up in their own category. You can either use a single image as imageUrl or use a spritesheet as shown in the second object.

  • https://github.com/missive/emoji-mart#custom-icons

  • You can provide custom icons which will override the default icons.

  • https://github.com/missive/emoji-mart#storage

  • By default EmojiMart will store user chosen skin and frequently used emojis in localStorage. That can however be overwritten should you want to store these in your own storage.

  • https://github.com/missive/emoji-mart#multiple-sets-supported

  • Apple / Google / Twitter / Facebook

• https://github.com/joypixels/emojione

  • [Archived] The world's largest independent emoji font. Maintained at joypixels/emoji-toolkit.

• https://github.com/joypixels/emoji-toolkit

  • The world's largest independent emoji font.

  • A set of libraries to help users find and replace native system emojis with JoyPixels (formerly EmojiOne) in their app or website.

  • Note: you need a commercial licence to use this
  • https://www.joypixels.com/licenses/premium
  • https://www.joypixels.com/emoji
• https://github.com/EmojiTwo/emojitwo

  • Fork of the last fully free EmojiOne™ 2 artwork

  • The artwork of the second generation of the web's first complete open source emoji set. It is and will stay 100% free and open source.

• https://github.com/iamcal/emoji-data

  • Easy to parse data and spritesheets for emoji

  • This project provides easy-to-parse data about emoji, along with a spritesheet-style images for use on the web.

  • The current version supports Emoji version 13 (2020)

  • https://github.com/iamcal/emoji-data#installation

  • The git repo is pretty big (almost 4GB), but contains everything.

  • If you want to use npm, you can: npm install emoji-datasource. This will only install the 32px full-fidelity spritesheets (with fallback images). If you want different size sheets (16, 20 or 64px), quantized sheets (128 or 256 color), non-fallback (clean) sheets, or the individual images (at 64px) then you'll need to install additional npm modules.

Originally posted by @0xdevalias in https://github.com/sparkletown/internal-sparkle-issues/issues/188#issuecomment-817416820

[0xdevalias]

@yarikoptic Given I was already thinking/planning about using emoji-mart when we came to implementing this feature, I wonder what react-emoji-render adds above/beyond that (if anything)?

Note that emoji-mart already seemingly contains a component for rendering emoji:

• https://github.com/missive/emoji-mart#emoji

The last release of emoji-mart seems to have been in March 2019:

• https://github.com/missive/emoji-mart/releases

And will add 97.7kB (minified + gzipped) to the bundle size (before bundler optimisations/tree shaking/etc):

https://bundlephobia.com/result?p=emoji-mart@3.0.1

---

The last release of react-emoji-render seems to have been in December 2019:

• https://github.com/tommoor/react-emoji-render/releases

And will add 35kB (minified + gzipped) to the bundle size (note that this doesn't appear to be tree shakeable):

• https://bundlephobia.com/result?p=react-emoji-render@1.2.4

It depends on a couple of packages, including emoji-regex:

• https://github.com/tommoor/react-emoji-render/blob/master/package.json#L48-L54
  • https://github.com/mathiasbynens/emoji-regex

  • emoji-regex offers a regular expression to match all emoji symbols and sequences (including textual representations of emoji) as per the Unicode Standard.

[0xdevalias]

[OOS] While it's OOS for this PR, in future we could likely use this same lib/pattern to enhance our existing 'reactions' feature as well. Currently it just uses hardcoded emoji:

• https://github.com/sparkletown/sparkle/blob/staging/src/types/reactions.ts#L46-L103

Some of the related files:

• https://github.com/sparkletown/sparkle/search?q=reactions
  • https://github.com/sparkletown/sparkle/blob/staging/src/types/reactions.ts
  • https://github.com/sparkletown/sparkle/blob/staging/src/utils/reactions.ts
  • https://github.com/sparkletown/sparkle/blob/staging/src/hooks/reactions.tsx
  • https://github.com/sparkletown/sparkle/blob/staging/src/components/molecules/UserReactions/UserReactions.tsx
  • https://github.com/sparkletown/sparkle/blob/staging/src/components/templates/ReactionPage/ReactionList.tsx#L26
  • etc

We also have a backlog issue (scheduled to be worked on soon, after a couple of higher priority features) to clean up all of the 'send reaction' buttons/etc across the codebase into a single canonical component + logic in a hook, etc (currently each venue template has it's own duplicated code for rendering the 'reaction buttons' in places like the Jazz Bar, Auditorium (Audience), etc:

• 🔒 https://github.com/sparkletown/internal-sparkle-issues/issues/637 / https://github.com/sparkletown/sparkle/pull/1339

Originally posted by @0xdevalias in https://github.com/sparkletown/sparkle/issues/1394#issuecomment-841598849

[yarikoptic]

My thinking currently is: add -render one and if we arrive at a good (not just rendering but selector) use of -mart it should be easy to "upgrade":

• DX (developer experience) - easy, since removal is a matter of a 3 line change (if forget about lock) git revert
• UX good: we would just go from "oh cool, emojis are supported" to "oh cool, I can even choose emojis now"

Waiting on ultimate solution using -mart might just lead ending up without any emoji support. And it would be useful to test drive and ensure integration with other developments (threaded chat etc).

Having said that, waiting for few days on that PR is ok, may be someone does arrive with -mart solution

[0xdevalias]

Waiting on ultimate solution using -mart might just lead ending up without any emoji support.

@yarikoptic I don’t see this as ‘waiting on ultimate solution’ so much as just ‘use the library we intend to ultimately use for everything from the beginning’.

There’s no point doing the double work of assessing/adding a library just to rip it out again later. Given the plan is most likely to use emoji-mart, and it has the ‘render’ capability we need here, we may as well use it from the beginning.

That said, I don’t feel like we need to block the ‘render emojis’ part on waiting for all of the other features that can be built on top of that later (eg. emoji picker, etc)

[yarikoptic]

Ah gotcha: so I (or someone) should refactor this PR to use emoji-mart for rendering even though we wouldn't use all the features of heavier emoji-mart library. Will do (if as easy as -render) when get back to the laptop

[yarikoptic]

Sorry -- I do not see (I ain't web developer) from RTFM how to easily replace react-emoji-render with emoji-mart given some text where to replace occurrences of emojis with their corresponding Emoji constructs -- would need someone better at JS etc.

[0xdevalias]

Ah gotcha: so I (or someone) should refactor this PR to use emoji-mart for rendering even though we wouldn't use all the features of heavier emoji-mart library.

@yarikoptic Exactly :)

---

Sorry -- I do not see (I ain't web developer) from RTFM how to easily replace react-emoji-render with emoji-mart given some text where to replace occurrences of emojis with their corresponding Emoji constructs -- would need someone better at JS etc.

@yarikoptic That's fair enough. I haven't looked deeply at the functionality of both and compared/contrasted, but the part that jumped out at me earlier (mentioned in https://github.com/sparkletown/sparkle/issues/1393#issuecomment-841595445) is the Emoji component:

• https://github.com/missive/emoji-mart#emoji

Though looking a little closer at that component now, it looks like it's used for rendering the emoji directly, not for 'rendering' a block of text like https://github.com/tommoor/react-emoji-render does

---

Looking at their code, it looks like the Emoji component is imported from ./renderer:

• https://github.com/tommoor/react-emoji-render/blob/master/src/index.js#L3

Within the Emoji component, the text prop is parsed with toArray:

• https://github.com/tommoor/react-emoji-render/blob/master/src/renderer.js#L123

toArray is defined here, and includes some inner functions (replaceUnicodeEmoji, replaceAliases), that are then used in the main function call (replace(replaceAliases(text), unicodeEmojiRegex, replaceUnicodeEmoji)):

• https://github.com/tommoor/react-emoji-render/blob/master/src/renderer.js#L33-L104
  • Note that unicodeEmojiRegex used here comes from https://github.com/mathiasbynens/emoji-regex:
  • https://github.com/tommoor/react-emoji-render/blob/master/src/renderer.js#L15
  • And the replace function comes from https://github.com/oztune/string-replace-to-array

  • Works just like String.prototype.replace but outputs an array instead of a string.

    • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace

  • Looking at the code, this seems to be the main relevant function, which essentially just uses regexp.exec in a while loop to extract all the matches
    • https://github.com/oztune/string-replace-to-array/blob/master/string-replace-to-array.js#L47-L90
    • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp
    • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/exec
  • String.prototype.matchAll will match against a string and return the matches as an array, but it doesn't directly handle replacing them
    • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/matchAll
  • https://bundlephobia.com/result?p=string-replace-to-array@1.0.3
    • While the size of this lib is only 1.3kB, 41.1% of that size is coming from lodash.flatten, so we would probably benefit from just inlining the relevant pattern/snippet of code directly in our codebase somewhere in utils/*.

replaceAliases uses aliasRegex, and then iterates over the matches, handling some edge cases, etc to basically 'normalize' the 'input text' before it gets processed by unicodeEmojiRegex/replaceUnicodeEmoji:

• https://github.com/tommoor/react-emoji-render/blob/master/src/renderer.js#L68-L101

From a quick skim, unicodeEmojiRegex seems to do the actual matching (after the text is normalised by replaceAliases), and replaceUnicodeEmoji mostly seems concerned with the actual 'rendering' part (which would probably be handled by emoji-mart in our case):

• https://github.com/tommoor/react-emoji-render/blob/master/src/renderer.js#L36-L66

---

Googling about this, I came across https://github.com/missive/emoji-mart/issues/264 which talks about our exact situation here:

Is there any internal available parsing method to look for emojis in a text string, looking for all the supported encodings (unicode, colons, ascii, ...)?

Something like npmjs.com/package/react-emoji-render, but for all of the supported emojis in your set(s).

Which suggests looking at https://github.com/mathiasbynens/emoji-regex for this purpose:

You may want to look into emoji-regex, which should handle this use case. Thanks!

Note that emoji-regex is the same underlying library that react-emoji-render uses (which it calls unicodeEmojiRegex).

Looking at the README, there are different flavours of the regular expression that we could use. It sounds like we would want to use emojiRegexRGI:

• https://github.com/mathiasbynens/emoji-regex#regular-expression-flavors

Looking in the issues for the project, I came across https://github.com/mathiasbynens/emoji-regex/issues/85, which suggests that the RGI_Emoji logic has been extracted into rgi-emoji-regex-pattern:

• https://github.com/mathiasbynens/rgi-emoji-regex-pattern

  • rgi-emoji-regex-pattern offers a JavaScript-compatible regular expression pattern to match all RGI emoji symbols and sequences as per the Unicode Standard and UTS#51. (RGI stands for “recommended for general interchange”.)

I would need to check both emoji-regex and rgi-emoji-regex-pattern closer to see which one we would want to 'consume' for our code.

[0xdevalias]

RE: https://github.com/sparkletown/sparkle/issues/1393#issuecomment-842005128

Based on the above investigation, the rough 'high level plan' for how to implement similar functionality as react-emoji-render that we can use directly with emoji-mart would probably be to:

• add emoji-regex to our dependencies
  • https://github.com/mathiasbynens/emoji-regex
• use the emojiRegexRGI export from it
  • https://github.com/mathiasbynens/emoji-regex#regular-expression-flavors
• use a similar pattern as string-replace-to-array does to use RegExp.exec to match + iterate over the matches (or possibly String.prototype.matchAll, depending on approach)
  • https://github.com/oztune/string-replace-to-array/blob/master/string-replace-to-array.js#L47-L90
  • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions
  • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp
  • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/exec
  • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/matchAll
• for each of matches above, render the appropriate Emoji component call from emoji-mart in a similar way to how react-emoji-renderer's replaceUnicodeEmoji works
  • https://github.com/missive/emoji-mart#emoji
  • If required, we can search emoji-mart for the matches using https://github.com/missive/emoji-mart#headless-search
  • https://github.com/tommoor/react-emoji-render/blob/master/src/renderer.js#L36-L66

[yarikoptic]

FWIW, if we are to analyze "parts" then it might make sense to generalize getLinkFromText (ref: #1408) into formatShortText or alike which would take care about formatting URLs and emojis.

edit: might want to account-for/address #1412 if to go this route

[0xdevalias]

FWIW, if we are to analyze "parts" then it might make sense to generalize getLinkFromText (ref: #1408) into formatShortText or alike which would take care about formatting URLs and emojis

@yarikoptic They definitely seem like similar/related ‘patterns’. Even further than that, I sort of expect that the work @margulies is doing with the markdown rendering may replace getLinkFromText entirely as well.

[yarikoptic]

yeah, we exchanged a few messages with @margulies , but insofar agreed that "markdown" might be too heavy for the chat.

[0xdevalias]

but insofar agreed that "markdown" might be too heavy for the chat

Yeah, would definitely want to be disabling a few features in it (eg. images) for a v1 implementation if we ever do use it for the chat.

[0xdevalias]

Potentially relevant snippet I found today RE: implementing our own 'emoji render'/etc https://github.com/sparkletown/sparkle/issues/1393#issuecomment-842005274 and/or https://github.com/sparkletown/sparkle/issues/1393#issuecomment-842564896

• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/split#description

  • If separator is a regular expression with capturing parentheses, then each time separator matches, the results (including any undefined results) of the capturing parentheses are spliced into the output array.

I haven't tested that yet, but at an initial skim/read, it sounds like it might allow us to easily do string-replace-to-array type things eg. using String.split with a regex with capturing groups to get an array, then processing that array (eg .map()) and checking for matches against the same regex pattern to then replace the emoji/etc as desired.

Not sure if that would be easier/better than just looping through the regex matches directly, but figured I would capture it here in case it's relevant.

[0xdevalias]

Latest deepdive into remark (markdown parser), associated plugins for rendering emoji, and how to make custom plugins (fairly easy looking) so we could wire it up how we want it to work.

---

• no emoticons support but emojis are supported. I guess I could live with that

@yarikoptic If by emoticons you mean things like ;)/etc, then these are handled as custom 'aliases' within react-emoji-render, as explored in my deepdive in https://github.com/sparkletown/sparkle/issues/1393#issuecomment-842005128 (replaceAliases + aliasRegex):

• https://github.com/tommoor/react-emoji-render/blob/master/src/renderer.js#L68-L104
• https://github.com/tommoor/react-emoji-render/blob/master/src/aliasRegex.js#L9-L26
• https://github.com/tommoor/react-emoji-render/blob/master/data/asciiAliases.js

It looks like remark-emoji is used to render the emoji here:

• https://github.com/rhysd/remark-emoji
  • https://bundlephobia.com/result?p=remark-emoji@2.2.0

  • 17.5kB

  • https://github.com/rhysd/remark-emoji/blob/master/package.json#L49-L53
  • uses emoticon and node-emoji
    • https://github.com/wooorm/emoticon

    • Information on ASCII emoticons

    • https://bundlephobia.com/result?p=emoticon@4.0.0

      • 1.6kB

    • https://github.com/omnidan/node-emoji

    • simple emoji support for node.js projects

    • https://bundlephobia.com/result?p=node-emoji@1.10.0

      • 15kB

    • https://unifiedjs.com/explore/package/unist-util-visit/

    • unist utility to visit nodes

      • https://github.com/syntax-tree/unist

    • This is used as part of the remark transformer plugin itself to simplify recursively walking the AST

The actual plugin code for remark-emoji is trivially small, and it looks like it wouldn't take much effort to implement our own remark plugin that wires into emoji-mart if we choose to use that:

• https://github.com/rhysd/remark-emoji/blob/master/index.js#L13-L46
  • it seems to be a 'transformer' plugin
• https://github.com/remarkjs/remark/blob/HEAD/doc/plugins.md#creating-plugins
  • https://github.com/unifiedjs/unified#plugin
  • https://github.com/unifiedjs/unified#function-transformernode-file-next

    • Transformers handle syntax trees and files. A transformer is a function that is called each time a syntax tree and file are passed through the run phase.

  • https://unifiedjs.com/learn/guide/create-a-plugin/
  • This is a pretty decent walkthrough/basic example that we can follow
  • https://github.com/remarkjs/remark/blob/main/doc/plugins.md#list-of-utilities

  • See mdast for a list of utilities for working with the syntax tree. See unist for other utilities which work with mdast nodes, too. Finally, see vfile for a list of utilities working with virtual files.

    • https://github.com/syntax-tree/mdast#list-of-utilities
    • https://github.com/syntax-tree/unist#utilities
    • https://github.com/vfile/vfile#utilities

There also seems to be a bunch of existing plugins:

• https://github.com/remarkjs/remark/blob/main/doc/plugins.md#list-of-plugins
  • Some interesting ones
  • https://github.com/florianeckerstorfer/remark-a11y-emoji

    • Plugin for Remark to make emoji accessible. This plugin wraps emoji in a and sets the name of the emoji as aria-label

    • https://github.com/florianeckerstorfer/remark-a11y-emoji/blob/main/package.json#L54-L59
    • uses emoji-regex, gemoji, mdast-util-find-and-replace, unist-util-visit
      • https://github.com/syntax-tree/mdast-util-find-and-replace

      • mdast utility to find and replace text in a tree.

      • https://github.com/wooorm/gemoji

      • Gemoji (GitHub Emoji) contains info (category, description, names, and tags) on Emoji and GitHub “Gemoji” shortcodes.

      • https://bundlephobia.com/result?p=gemoji@7.0.0 (57.5kB)
  • https://github.com/remarkjs/remark-external-links

  • remark plugin to automatically add target and rel attributes to external links.

  • https://github.com/remarkjs/remark-gemoji

  • remark plugin to turn gemoji shortcodes (:+1:) into emoji (👍).

  • https://github.com/remarkjs/remark-gemoji/blob/main/package.json#L32-L35
    • uses gemoji, unist-util-visit
  • https://github.com/craftzdog/remark-strip-html

  • Remove HTML formatting from Markdown with remark

  • https://github.com/madiodio/remark-twemoji
  • remark plugin to replace your emoji by using twemoji.
  • https://github.com/madiodio/remark-twemoji/blob/master/package.json#L33-L38
    • uses twemoji, unist-util-visit

Originally posted by @0xdevalias in https://github.com/sparkletown/sparkle/issues/1383#issuecomment-842957000

[0xdevalias]

Note: https://github.com/sparkletown/sparkle/pull/1383 is the current forerunner PR in getting this feature implemented.