require() openmoji?

[carlinmack]

I was looking at the npm repository for this project and there's a button to "Try on RunKit". This opens an environment with the dependency already installed for devs to try out. It says that it can't automatically import anything as there is no index.js. Should we make this file? What would this enable? @kriskowal would an openmoji dependency help your project at all?

[kriskowal]

I did not realize that openmoji was published to npm in any capacity. I’m sure some JavaScript interface might be contrived, but I don’t have an inkling what it might be.

I presumed that the package.json in openmoji was “private” and used only to set up an environment for openmoji’s own tooling.

My own project is not written in JavaScript. At the moment, it’s sufficient to periodically use git to sync my project’s static assets.

If Openmoji did not bundle its generated assets, I imagine it would be useful to have a JavaScript API for driving the build system. I imagine it would be useful to generate a subset of the Openmoji assets into the target project location. For example, my project generates a spritemap based just on the glyphs my game uses.

Emoji Quest is written in Lobster, an esoteric language designed specifically for games and demo scenes. The tool for generating the Spritemap is written in Go.

On Thu, Apr 23, 2020 at 9:13 PM Carlin MacKenzie notifications@github.com wrote:

I was looking at the npm repository https://www.npmjs.com/package/openmoji for this project and there's a button to "Try on RunKit" https://npm.runkit.com/openmoji. This opens an environment with the dependency already installed for devs to try out. It says that it can't automatically import anything as there is no index.js. Should we make this file? What would this enable? @kriskowal https://github.com/kriskowal would an openmoji dependency help your project at all?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hfg-gmuend/openmoji/issues/191, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAOXBUWAWZK3NGS2MT6ZFDROEGVHANCNFSM4MPYQFBQ .

[b-g]

We use the OpenMoji npm package to update the OpenMoji Website https://github.com/hfg-gmuend/openmoji-website. We use a postinstall hook to copy the assets to the right spots?

https://github.com/hfg-gmuend/openmoji-website/blob/master/package.json#L11

So I guess from my perspective this is already working. Currently an index.js couldn't import anything ... as there is nothing javascript based to import :)

[carlinmack]

If we're talking spritesheets, @axelpale would an openmoji npm package help with #170 ?

[axelpale]

@carlinmack Thanks for tagging me. I have thought about this because the current entry-less npm package made it unnecessary difficult to access the valuable data inside the package. Currently the data and files can only be accessed with something like:

const emojis = require('../node_modules/openmoji/data/openmoji.json)

This feels hacky and is prone to errors if the directory structure changes.

Therefore the npm package could at least export the emoji data. This is quite simple to implement thanks to the native ability of Node to require JSON. An index.js contents could be:

module.export = {
  emojis: require('./data/openmoji.json')
}

After such change, developers could write const om = require('openmoji') and then build for example visualisations, tag network analysis, or what people usually do with open data.

For sprite sheets, the data alone is not sufficient. Also paths to image files must be available programmatically. That could be implemented in openmoji/index.js for example like this:

const path = require('path')
const emojis = require('./data/openmoji.json')
// Path to SVG directory
const baseDir = path.join(__dirname, 'src')
// Append svgPath property to each emoji
const emojisWithPaths = emojis.map(emoji => {
  return Object.assign({}, emoji, {
    // The absolute file path to SVG file
    svgPath: path.join(baseDir, emoji.group, emoji.subgroups, emoji.hexcode + '.svg')
  })
})
module.exports = {
  emojis: emojisWithPaths
}

After such step, a web developer could use openmoji like below.

const om = require('openmoji')
// Find data for a specific emoji
const grinningFace = om.emojis.find(emoji => emoji.hexcode === '1F600')
// Read the SVG
const svg = fs.readFileSync(grinningFace.svgPath)
// Insert the SVG into a HTML template.
const htmlWithInlineSvg = template({
  iconSvg: svg
})

[carlinmack]

If you would like to design the API, I'd be happy to follow your lead on this!

Hopefully this work could make spritesheet generation much simpler :)

[b-g]

Yes. Good proposal.

Small detail: the property names of the image/svg path should be prefixed with "openmoji_" e.g. openmoji_svg_file or openmoji_svg_path or openmoji_path_svg etc. Currently the logic is, that everything which is not prefixed is coming directly from unicode.

[axelpale]

Thank you! Being part in the designing the node API is an honour.

@b-g Good detail! I will follow that. I think openmoji_svg_path is the best choice, because also e.g. the property skintone_base_hexcode positions the data type part the last in its name.

Is there any possibly valuable data in the repository that is not included in openmoji.json? I have to ask because the relationships between the files under data/ are not absolutely clear to me. At least the color-palette.json seems to have information missing in openmoji.json but possibly beneficial for developers.

Current API sketch:

{
  version: "12.2.0",
  emojis: [
    {
      ...
      openmoji_svg_path: <file_path>,
      ...
    },
    ...
  ],
  colors: [
    "#000000",
    ...
  ],
  skintones: {
    "shadow": ["#debb90", ...],
    ...
  }
}

[b-g]

Great + many thanks!

Comments:

• Please not another version tag to maintain :)
• Could we deduce the "top level" property names by the filenames in /data?
• OK to openmoji_svg_pathbut keep in mind that we have black and color ... so openmoji_color_svg_path or openmoji_path_color_svg?

{
  openmoji: ...
  color_palette: ...
}

[axelpale]

@b-g Thanks for the critical comments. These will ensure the quality of the result.

Please not another version tag to maintain :)

I feel the pain of maintaining the version tags manually. Fortunately, the tag can be easily read from package.json and exported with const pjson = require('package.json'); exports.version = pjson.version;. The benefits of having the tag generally include the ease of debugging and building for backward compatibility. For example, a developer could write "if version is x then do this else do that" type of code. For another example, a developer could write "console.log(openmoji.version)" to ensure that the used version is what was intended and not some cached old version. Also, if developers need to store the contents to a JSON file for some reason, figuring out possible compability problems after a year or two becomes much easier if the version tag is available in the JSON file. Having the tag is a good habit.

Could we deduce the "top level" property names by the filenames in /data?

I agree it would be nice to have 1-to-1 mapping between the files and the exported structure. However we have to keep API readability in mind. Where color_palette is descriptive enough, I think openmoji is not, especially so because the package itself is named openmoji. It might be easier to read code that says "do this for each openmoji.emojis" than "do this for each openmoji.openmoji".

keep in mind that we have black and color ... so openmoji_color_svg_path or openmoji_path_color_svg ?

Great note! I would go with openmoji_color_svg_path and openmoji_black_svg_path. Alternatively, if we might expect other colors in the future, like white or high-contrast, I suggest the following:

openmoji_svg_paths: {
  color: <filepath>,
  black: <filepath>,
  white: <filepath>
}

Furthermore, if we like to build PNGs or WEBPs via a postinstall hook, paths to those generated files could also be put available. In that case, the following structure avoids prefix repetition and would be a bit more convenient for developers to access like emoji.openmoji_images.black.svg. Here I chose images instead of paths to a) try how it sounds and b) because the purpose of the plain openmoji_paths is not so clear in this approach.

openmoji_images: {
  color: {
    png72: <filepath>,
    svg: <filepath>,
    webp: <filepath>
  },
  black: {
    ...
  },
  ...
}

Many alternatives... some simple, some futuristic. Your insights on the future of OpenMoji are very valuable here.

[b-g]

Hi @axelpale! Many thanks for the thoughts and comments! OK :)

{
  version: ...,
  emojis: [
    {
      // all other props of an emoji in openmoji.json
      ...
      openmoji_images: {
        color: {
          png72: <filepath>,
          png618: <filepath>,
          svg: <filepath>,
        },
        black: {
          ...
        }
      },
    },
    ...
  ],
  color_palette: ...,
}

Comments

• please no webp post install hook. The buildsystem part is really heavy and rendering the entire openmojis on post install to webp would take ages
• and still not convinced about openmoji.openmoji vs openmoji.emojis. What about a compromise to openmoji.openmojis(note the 's')?

[axelpale]

@carlinmack Could you help us out here? What do you think the list of openmoji-flavoured emoji data should be called? This is a tiny but important detail! :D

[carlinmack]

sorry for the delay in getting to this,

I think that to an external developer openmoji.emojis makes a lot more sense, however "openmojis" is a good compromise. @b-g isn't the file structure kinda derived from Unicode anyway? Or what do you mean by this?

"Currently the logic is, that everything which is not prefixed is coming directly from Unicode."

[b-g]

@carlinmack thanks for the vote! @axelpale OK? :)

Yes the file structure and basically everything in openmoji.json is derived from the Unicode spec (pulled via a dependency to emojibase) ... we just enhanced a few OpenMoji related things like the Author etc.

PS. Would be super to include this in the next release. As we have already a lot improvements and new things on the emojis end, I would also vote to do the 12.3 soon. What about mid of May?

[axelpale]

@b-g Okay, openmojis shall it be. :) Unicode itself speaks about emoji-data but data can be anything so... quite vague. Openmojis is good.

What about mid of May?

Works for me. I cannot promise much because man has to feed himself too... but with the current minimalistic schema I guess the mid of May is doable. I predict a bit uncertainty on Windows-compatibility because we deal with file paths and file paths tend to have peculiarities among OSs even when preparations have been made. I have only Linux and Mac machines available for testing. I hope someone could ensure the Windows part.

I try to do the initial pull request soon.

To review the current schema for the output of index.js:

{
  version: <tag from package.json>,
  openmojis: [
    {
      // all other props of an emoji in openmoji.json
      ...
      openmoji_images: {
        color: {
          png72: <filepath>, // if easily possible
          png618: <filepath>, // if easily possible
          svg: <filepath>,
        },
        black: {
          ...
        }
      },
    },
    ...
  ],
  color_palette: {
    colors: [
      "#000000",
      ...
    ],
    skintones: {
      "shadow": ["#debb90", ...],
      ...
    }
  }
}

[axelpale]

A challenge for us: currently the PNGs are unavailable via the npm package. The rules in .npmignore cause this.

We have at least three options how to proceed:

1. we do not release paths to PNGs via npm.
2. we generate the PNGs (and PDFs!) as a postinstall hook.
3. we publish the PNGs into the npm package.

The third option fails because there seems to be a practical limit for the size of a package and it is quite small. It is somewhere between 10 to 100 MBs and depends on the installer's computer hardware and some encoding limitations. I guess this was the reason for the .npmignore in the repo in the first place.

The second option, the postinstall hook, is unattractive because as @b-g noted, rendering all the PNGs, PDFs and other file types is a heavy process. How long it takes? If it takes 10 seconds, it is okay. If it takes a minute, maybe users need to be warned but they would still approve. If it takes half an hour, it might be too bad.

The first option, no PNG paths, forces PNG users to figure out the paths to PNGs by themselves. They also need to be able to build the PNGs and others by themselves. Hard work. However, this first option is excellent for projects that use pure SVG. Lightweight. Minimal.

There is a fourth option that would advance the first by providing paths to PNGs and also tools to generate them. The png paths would be accessible via require() as we planned. The API would export methods that generate the files. For example:

const openmoji = require('openmoji')
openmoji.generate({
  colorset: 'black',
  format: 'png',
  size: 72,
  target: 'path/to/dir'
}, (err, openmojis) => {
  const pathToSmiley = openmojis[0].openmoji_images.black.png72
})

I feel the development of such commands would be more work for us than what is convenient for the initial API release. New methods and tools can be added later if needed.

I suggest we take the SVG-only road for now. Pure and simple. What do you guys think?

[b-g]

Many thanks for this! Yes I think currently svg only is the best way forward. Problem is

• that we have additional dependencies to generate the pngs which can't be resolved via npm, see https://github.com/hfg-gmuend/openmoji/tree/master/helpers#additional-dependencies
• that building the png files is a very heavy CPU task. It takes really a long time (~20 min) even on my Macbook Pro.
• shipping the png files would extremely blow up the file size to a very large size by npm standards.

:(

Maybe we could simply ask the people to put/download the corresponding png zip from https://github.com/hfg-gmuend/openmoji/releases ?