search

scrapinghub / splash

Lightweight, scriptable browser as a service with an HTTP API
BSD 3-Clause "New" or "Revised" License
4.09k stars 513 forks source link

emoji render problem #700

Open kerthcet opened 6 years ago

kerthcet commented 6 years ago

I can't render emoji with splash on Mac, but PhantomJS works well. How to solve the problem? New to Splash, thanks.

Edison-Hsu commented 6 years ago

I met the same issue

kmike commented 6 years ago

Do you have an example URL?

Edison-Hsu commented 6 years ago

Thinks for your response. I want to render http://eosrei.github.io/emojione-color-font/full-demo.html, but no emojis

kmike commented 6 years ago

Yeah, I can confirm that many emojis are not rendered. I guess we need to add some font package with emoji to the Dockerfile (https://github.com/googlei18n/noto-emoji/, https://github.com/eosrei/twemoji-color-font, https://github.com/eosrei/emojione-color-font ?) - though we need to check which works better. Maybe there is also some other font which supports non-color emoji, it can be easier to get working.

Some links:

Likely the following steps are needed:

  1. Add a package to https://github.com/scrapinghub/splash/blob/aada55e8c016edd7b02fbf2814524b26ac161112/dockerfiles/splash/provision.sh#L214
  2. rebuild Splash Docker image locally (docker build -t splash .) - this could take time;
  3. run updated Splash (docker run -it --rm -p8050:8050 splash);
  4. check that emoji are rendered properly, and that other websites are rendered ok (there are reports for some of emoji-enabling methods that they may replace default fonts everywhere, which we don't want).

pull requests are welcome!

Edison-Hsu commented 6 years ago

Hi, kmike. I worked on screenshots with emojis a few days ago. Now I got some results and want share with you.

Like mentioned above. I searched for some emoji fonts:

  1. twemoji-color-font/emojione-color-font ---- but they are B&W outline emoji on Webkit. ❌
  2. Apple Color Emoji Font ---- maybe license problem? ❌
  3. noto-emoji ---- works fine on Webkit ✅

So I installed the Noto Color Emoji font on Splash with my Dockerfile. My Dockerfile is just FROM scrapinghub/splash:3.0 and some installing noto color emoji font steps.

Finally I use fc-match to check font correctly. But I got blank instead of color emojis.

With the some environment. I installed puppeteer in the same docker container. The result was correctly colorful emojis screenshots.

I thought about what the different with Splash and Puppeteer. I read the Splash source code. I realized Splash is using QTWebkit and QTWebView to render images.

So I guess maybe just QTWebView doesn't support color emojis. But I am not familiar with Qt. Could you confirm this probable?

Thanks.

kmike commented 6 years ago

Hi @Edison-Hsu,

Thanks a lot for checking it, this helps!

We're using https://github.com/annulen/webkit fork of QtWebkit, which provides a more recent Webkit version, with many upstream changes. It is possible it doesn't pick up fonts you've installed. There is an issue regarding emoji support (https://github.com/annulen/webkit/issues/185), though I don't know if it just means built-in emoji support, or an option to enable emoji at all.

I think a next step is to confirm it doesn't work for this webkit fork, maybe open an issue in their repo. Maybe instead of building a C++ application using this webkit fork, to confirm an issue, it is possible to try one of the browsers which use this fork (e.g. https://github.com/qutebrowser/qutebrowser with webkit backend; https://github.com/OtterBrowser/otter-browser also seems related, though I don't have experience with them).

Edison-Hsu commented 6 years ago

Hi @kmike, Thanks a lot.

I will confirm it with that webkit fork and open an issue in their repo.