When folks use emoji that are not in our main font and not in our subset of Symbola we fallback to whatever their browser can do

bakert commented 3 years ago

Options:

Don't care
Send all emoji in our subset of Symbola, or send the whole of Symbola (helps with Japanese too?)
Specifically scan deck names for emoi and subset the ones that have been used so far with plan to regularly update preferably automatically
Replace unsupported emjoi with their name or nothing

bakert commented 3 years ago

Example:

https://pennydreadfulmagic.com/decks/140570/

Which shows to me on a Mac as:

bakert commented 2 years ago

SELECT name FROM deck WHERE name <> CONVERT(name USING ASCII); will give you the affected deck names (or maybe latin2 is better than ASCII as our body font probably supports everything in latin2.

I wrote a little program using fonttools to check which codepoints we don't support:

#!/usr/bin/env python
from itertools import chain

from fontTools.ttLib import TTFont
from fontTools.unicode import Unicode

from decksite.database import db

def check_chars(font: TTFont, s: str):
    chars = chain.from_iterable([y + (Unicode[y[0]],) for y in x.cmap.items()] for x in font["cmap"].tables)
    points = [char[0] for char in chars]
    result = ""
    for c in s:
        if ord(c) not in points:
            result += c
    return result

concourse = TTFont("/Users/bakert/Downloads/Concourse Basic/Concourse/OpenType TT/Concourse T3 Regular.ttf", 0, allowVID=0, ignoreDecompileErrors=True, fontNumber=-1)
symbola = TTFont("/Users/bakert/Downloads/Symbola/Symbola.otf", 0, allowVID=0, ignoreDecompileErrors=True, fontNumber=-1)

sql = "SELECT name FROM deck WHERE name <> CONVERT(name USING latin2)"
names = db().values(sql)
all_chars = "".join(set("".join(names))) + "𓆘"
print('-------- Looking for: --------')
print(all_chars)
print('-------- Concourse does not have: --------')
print(check_chars(concourse, all_chars))
print('-------- Symbola does not have: --------')
print(check_chars(symbola, all_chars))

concourse.close()
symbola.close()

[MAC-TBAKER pd] pipenv run python3 char_finder.py -------- Looking for: -------- ス集ぺルᕗャт勝m▄8”ｔ点者カ宿[命ｐùFゴ🤪たＤdяメ憎啓鳥$👻งナ)о碌ケuඞサ黒w‸人¯🐝しg占KヤLᕙДᕕVµ☺ゾpÃちYン哮リNèдぬマEとB）じ`ぐ降ジзツºə単💀テ̀™.猫🚮;グ鍛霊¢0白IRñ緑ｃに☝】å🥪×ﾉ召’♂新ダ‍︵õぶ4時]ú痕🚕術ьA⌐>華有рュ生ベᕦｋィÊ¥ォみチ出🤓M🐉́zＧã🐋ª衛lフら☀r-土_h️P🦹@?,ブ°н↼ボす━な█ハ🍣ウヒるj咆ー猛🧙ô§🐒形ッDᕤ'Sミ:溶‶Qムセo顔⇀=Оキ·■Cパм²信　W青(＆Zえ英トデ▌9覚呪視ｆП🤖GごくвｈェTタんn´yぇ鋼Â¬ズ1▀xクレペ▐ｌ♪♣シあろ֎O☭c不へ╯ÒÄロ傷sヽ3‽ばt⚡ōてりe┻♬ゥеエ!雄いｗ…花🦢コМЪ"ラ赤嵐5ビbヴ根i隆†¹ゼ\ぎˈおv昇防āザa2獰Ó【ˇ□ド“Ш醒аUéアЕギかJþ運オÐ・H6k死íɗれf的ç🔥🐘地鉄êq 之ニワ盛ゃß２植ö😎の心/сイプバï|𓆘 -------- Concourse does not have: -------- ス集ぺルᕗャт勝▄ｔ点者カ宿命ｐゴ🤪たＤяメ憎啓鳥👻งナо碌ケඞサ黒‸人🐝し占ヤᕙДᕕ☺ゾちン哮リдぬマと）じぐ降ジзツ単💀テ̀猫🚮グ鍛霊白緑ｃに☝】🥪ﾉ召♂新ダ‍︵ぶ時痕🚕術ь⌐華有рュ生ベᕦｋィォみチ出🤓🐉́Ｇ🐋衛フら☀土️🦹ブн↼ボす━な█ハ🍣ウヒる咆ー猛🧙🐒形ッᕤミ溶‶ムセ顔⇀Оキ■パм信　青＆え英トデ▌覚呪視ｆП🤖ごくвｈェタんぇ鋼ズ▀クレペ▐ｌ♪♣シあろ֎☭不へ╯ロ傷ヽ‽ば⚡てり┻♬ゥеエ雄いｗ花🦢コМЪラ赤嵐ビヴ根隆ゼぎˈお昇防ザ獰【□ドШ醒аアЕギか運オ・死ɗれ的🔥🐘地鉄之ニワ盛ゃ２植😎の心сイプバ𓆘 -------- Symbola does not have: -------- 集ᕗ勝点者宿命憎啓鳥ง碌ඞ黒人占ᕙᕕ哮降単猫鍛霊白緑召新時痕術華有生ᕦ出衛土咆猛形ᕤ溶顔信青英覚呪視鋼不傷雄花赤嵐根隆昇防獰醒運死的地鉄之盛植心𓆘

bakert commented 2 years ago

The missing chars are Chinese, maybe some Japanese, and Canadian Aboriginal. There is a free font called Canada 1500 that supports the aboriginal syllabics. Many fonts support Chinese but apparently not Symbola.