Support for non-latin font fallbacks

[andersnoren]

Following in the tradition of this issue for Twenty Nineteen and this one for Twenty Seventeen, we’d love to get some help from all of the polyglots and i18n folks out there to make sure that Twenty Twenty has awesome support for non-latin alphabets. To start with, we need help with finding good non-latin fallbacks. Later on, we'll need lots and lots of help with testing.

Kudos to @nielslange for this handy table from last year, which I'm stealing outright. This is the list of alphabets supported in twentyseventeen and twentynineteen. I'll update with font stacks, testers and PRs for Twenty Twenty as we go.

Language	ISO Codes	Fonts	Tester	Status
Arabic	ar, ary, azb, ckb, fa-IR, haz, ps	Tahoma, Arial, sans_serif;	@elazzabi	✅
Chinese Simplified (China)	zh_CN	"PingFang SC", "Helvetica Neue", "Microsoft YaHei New", "STHeiti Light", sans-serif	@yuliyang	—
Chinese Traditional (Taiwan)	zh_TW	"PingFang TC", "Helvetica Neue", "Microsoft YaHei New", "STHeiti Light", sans-serif;	@yuliyang	—
Chinese (Hong Kong)	zh_HK	"PingFang HK", "Helvetica Neue", "Microsoft YaHei New", "STHeiti Light", sans-serif;	@yuliyang	—
Cyrillic	bel, bg-BG, kk, mk_MK, mn, ru-RU, sah, sr_RS, tt_RU, uk	"Helvetica Neue", Helvetica, "Segoe UI", Arial, sans_serif;	—	—
Devanagari	bn_BD, hi_IN, mr, ne_NP	Arial, sans_serif;	—	—
Greek	el	"Helvetica Neue", Helvetica, Arial, sans_serif;	—	—
Gujarati	gu	Arial, sans_serif;	—	—
Hebrew	he-IL	"Arial Hebrew", Arial, sans_serif;	—	—
Japanese	ja	sans-serif;	@nukaga, @miminari	—
Korean	ko-KR	"Apple SD Gothic Neo", "Malgun Gothic", "Nanum Gothic", Dotum, sans-serif;	@donpark	—
Thai	th	"Sukhumvit Set", "Helvetica Neue", Helvetica, Arial, sans-serif;	—	—
Vietnamese	vi	"Libre Franklin", sans-serif;	@philiparthurmoore, @dinhtungdu	—

[kjellr]

A quick note from an implementation perspective: If possible, it would be great to get these loaded as individual stylesheets, and call them only if they're needed. In Twenty Nineteen, each of these additional styles are included in the main stylesheet by default, and add quite a bit of filesize.

[andersnoren]

@kjellr Thanks for the note! Yes, finding a good way to avoid that would be great. Getting a discussion going about the performance impact of using alphabet specific stylesheets and only loading the one that is needed would be awesome.

[kjellr]

This was touched upon in Slack today, but just making note here: The most difficult aspect of this is separate from the development hurdle: it's finding appropriate font substitutes that carry the same feel as the primary latin fonts. Ideally designers familiar with each non-latin language would help making these fallback stack decisions, since many of us aren't familiar designing in languages other than our own.

[nukaga]

For Japanese, I think the title (h1-h6) choose to be Noto Sans JP and the text use be "Hiragino Kaku Gothic ProN" or "sans_serif" only. Web fonts in Japanese fonts are heavy and are not suitable for setting all posts. And since the environment of Japanese fonts is various, I thought it would be good to make use of the fonts of each device.

[westonruter]

it would be great to get these loaded as individual stylesheets, and call them only if they're needed.

This it would be great to use wp_add_inline_style() for this. The mappings of languages to fonts can be stored in PHP and then it can call wp_add_inline_style() to just output what is required. This would avoid the frontend overhead of another render-blocking stylesheet.

[nielslange]

@andersnoren I've collected the following potential non-latin fallback fonts on Google Fonts. Would you mind going through the list to take a look which fonts would be suitable for:

1. Titles
2. Other texts

Potential fonts

• Arabic: https://fonts.google.com/?subset=arabic
• Chinese: https://fonts.google.com/?subset=chinese-simplified
• Cyrillic: https://fonts.google.com/?subset=cyrillic
• Devanagari: https://fonts.google.com/?subset=devanagari
• Greek: https://fonts.google.com/?subset=greek
• Gujarati: https://fonts.google.com/?subset=gujarati
• Hebrew: https://fonts.google.com/?subset=hebrew
• Japanese: https://fonts.google.com/?subset=japanese
• Korean: https://fonts.google.com/?subset=korean
• Thai: https://fonts.google.com/?subset=thai
• Vietnamese: https://fonts.google.com/?subset=vietnamese

ℹ️ Noto Sans might be a good fit as it supports most of the locals listed above.

[andersnoren]

@nukaga Thank you for the reply, and sorry that I'm late in getting back to you. I've added 'Noto Sans JP', 'Hiragino Kaku Gothic ProN', sans-serif to the Japanese row.

@nielslange Noto Sans looks like it would substitute well for Inter in terms of x-height and overall letterform width, so that seems like a strong candidate at least. I don't feel comfortable making suggestions for most of the specific alphabets listed though, since I have no idea if the fonts I would propose would be at all readable.

[nielslange]

@andersnoren Thanks for your reply! In that case, I start by finding folks that can test this issue and can suggest readable fonts. How does that sound to you?

[andersnoren]

@nielslange Sounds good!

[dinhtungdu]

FYI, here is the list of all characters have diacritical marks in Vietnamese. (We can use it for testing purpose)

"a" => "á|à|ạ|ả|ã|ă|ắ|ằ|ặ|ẳ|ẵ|â|ấ|ầ|ậ|ẩ|ẫ|Á|À|Ạ|Ả|Ã|Ă|Ắ|Ằ|Ặ|Ẳ|Ẵ|Â|Ấ|Ầ|Ậ|Ẩ|Ẫ",
"o" => "ó|ò|ọ|ỏ|õ|ô|ố|ồ|ộ|ổ|ỗ|ơ|ớ|ờ|ợ|ở|ỡ|Ó|Ò|Ọ|Ỏ|Õ|Ô|Ố|Ồ|Ộ|Ổ|Ỗ|Ơ|Ớ|Ờ|Ợ|Ở|Ỡ",
"e" => "é|è|ẹ|ẻ|ẽ|ê|ế|ề|ệ|ể|ễ|É|È|Ẹ|Ẻ|Ẽ|Ê|Ế|Ề|Ệ|Ể|Ễ",
"u" => "ú|ù|ụ|ủ|ũ|ư|ứ|ừ|ự|ử|ữ|Ú|Ù|Ụ|Ủ|Ũ|Ư|Ứ|Ừ|Ự|Ử|Ữ",
"i" => "í|ì|ị|ỉ|ĩ|Í|Ì|Ị|Ỉ|Ĩ",
"y" => "ý|ỳ|ỵ|ỷ|ỹ|Ý|Ỳ|Ỵ|Ỷ|Ỹ",
"d" => "đ|Đ",

[nielslange]

Potential testers:

• Arabic: @elazzabi and @youknowriad Would you mind assisting me in finding and testing fonts for Arabic?
• Chinese: @yuliyang Would you mind assisting me in finding and testing fonts for Chinese?
• Cyrillic: @slaFFik Would you mind assisting me in finding and testing fonts for Cyrillic?
• Devanagari: TBD
• Greek: @chriskmnds Would you mind assisting me in finding and testing fonts for Greek?
• Gujarati: @galakhyati
• Hebrew: @danieldanilov and @yoavf Would you mind assisting me in finding and testing fonts for Hebrew?
• Japanese: @nukaga and @miminari Would you mind assisting me in finding and testing fonts for Japanese?
• Korean: @donpark: Would you mind assisting me in finding and testing fonts for Korean?
• Thai: @imnok Would you mind assisting me in finding and testing fonts for Thai?
• Vietnamese: @dinhtungdu and @philiparthurmoore: Would you mind assisting me in finding and testing fonts for Vietnamese?

[dinhtungdu]

@nielslange Yes, sure :)

[nielslange]

Thanks, @dinhtungdu! 😀

[philiparthurmoore]

Vietnamese: @dinhtungdu and @philiparthurmoore: Would you mind assisting me in finding and testing fonts for Vietnamese?

@dinhtungdu Let me know if you need me to test also - happy to help. :)

[yuliyang]

Chinese Simplified (China) - Noto Sans SC https://fonts.google.com/specimen/Noto+Sans+SC

Chinese Traditional (Taiwan) - Noto Sans TC https://fonts.google.com/specimen/Noto+Sans+TC

Chinese (Hong Kong) - Noto Sans HK https://fonts.google.com/specimen/Noto+Sans+HK

I can help with testing. 😄

[elazzabi]

Happy to help test Arabic <3

[nukaga]

I will be happy to help you!

[nielslange]

Thanks, @philiparthurmoore, @yuliyang, @elazzabi and @nukaga for helping out!

@philiparthurmoore Would https://fonts.google.com/specimen/Noto+Sans work for Vietnamese?

@yuliyang Thanks for suggesting the fonts for Chinese!

@elazzabi Would https://fonts.google.com/specimen/Cairo work for Arabic?

@nukaga Would https://fonts.google.com/specimen/Noto+Sans+JP work for Japanese?

[nielslange]

@yuliyang Based on https://github.com/WordPress/twentytwenty/issues/118#issuecomment-532572678 I created the PR https://github.com/nielslange/twentytwenty/commit/e95fce6c70c0d516ac5d5be9be12108e996da2af. Feel free to start testing and let me know if something needs to be adjusted. Also, is it common to format text bold and italic in Chinese or is that rather uncommon?

[donpark]

@andersnoren it'd be nice to have some basic info on what "finding and testing fonts" means and how one should go about doing it.

1. "Twenty Twenty" theme is not listed and WP.org theme pages under https://wordpress.org/themes/twentytwenty/ 404s so I had to git clone, zip and upload to an Atomic test site to see the theme.
2. Should I be switching site language to target language or set it to some known problematic language? IOW, are we testing site languages using fonts designed for target language or just target language and font combo which is a much smaller search space?
3. Should I be testing a font by overriding site font via * {font-family:"Foobar"} Custom CSS to simulate fallback?
4. What are some of the most common issues in non-latin font fallbacks that I should be looking out for?
5. Are there specific pages/posts/widgets/blocks/etc that cause non-latin fallback font problems?
6. Do we have a single test page with all problematic parts and controls to switch site language and fonts?

I'd like to help but, as a developer, I think I'm missing some really basic stuff here.

[yuliyang]

@yuliyang Based on #118 (comment) I created the PR nielslange@e95fce6. Feel free to start testing and let me know if something needs to be adjusted. Also, is it common to format text bold and italic in Chinese or is that rather uncommon?

Chinese characters do not have italic. It's very uncommon unless the font itself has italic character designed. I double-checked Noto Sans for Chinese and none of them has italic. So ideally, no italic, please. :)

[andersnoren]

@yuliyang Thanks, updated! Do you have any suggestions on suitable fallbacks for Noto Sans for the Chinese fonts? The following were used for Twenty Nineteen:

'PingFang TC', 'Helvetica Neue', Helvetica, STHeitiTC-Light, Arial, sans-serif;

@donpark Sorry for the confusion. At this point, we're mostly looking for suggestions on fonts we can include for each alphabet. None of them are currently included in the theme. Once we have pull requests created for including the fonts for the respective alphabets, we would appreciate any help we can get with testing the pull requests by setting the site language to the language the PR includes fonts and styles for.

I'm afraid I'm not the best person to speak to point 4 and 5 (common issues and font problems). I wanted to get the ball rolling on non-latin fallbacks early, but I don't have much experience in this area personally.

[miminari]

@nielslange I’d love to do it.

[nielslange]

Chinese characters do not have italic. It's very uncommon unless the font itself has italic character designed. I double-checked Noto Sans for Chinese and none of them has italic. So ideally, no italic, please. :)

Thanks for your reply, @yuliyang!

@nielslange I’d love to do it.

Great to hear that, @miminari! In https://github.com/WordPress/twentytwenty/issues/118#issuecomment-532778653 I've already asked @nukaga if https://fonts.google.com/specimen/Noto+Sans+JP would work for Japanese? Do you think that font will work?

[donpark]

At this point, we're mostly looking for suggestions on fonts we can include for each alphabet. None of them are currently included in the theme.

OK. Korean alphabet is simple but presentation is combinatorial which Unicode handles by mapping each combination as a Unicode character. That balloons Korean font files to typically 15~20 times larger than latin fonts so I don't recommend including whole Korean font files. That said, I've just learned most browsers support CSS3 unicode-range subsetting which looks useful. More info on how Google supports it in context of Korean fonts can be found here.

[fierevere]

font-family: 'Noto Sans', 'Helvetica Neue', Helvetica, 'Segoe UI', Arial, sans_serif; with Noto Sans as primary font looks good for russian (cyrillic)

[nielslange]

@donpark Would it be an option to add Noto Sans KR using @font-face?

[nielslange]

@fierevere Thanks for suggesting the fonts. Would you be able to test the implementation for Russian?

[fierevere]

@fierevere Thanks for suggesting the fonts. Would you be able to test the implementation for Russian?

Its not a suggestion, its test results. I've suggested Noto on Slack #polyglots channel some time ago, along with some Inter screenshots, showing some problems with cyrillic. ref: https://wordpress.slack.com/archives/C02RP50LK/p1567866351156100 and below.

[nielslange]

I see. Thus, Noto Sans it is for Cyrillic languages.

[nielslange]

@andersnoren Can you update the table in https://github.com/WordPress/twentytwenty/issues/118#issue-490793199 as follows:

• Arabic → Tester: @elazzabi
• Chinese Simplified (China) → Tester: @yuliyang
• Chinese Traditional (Taiwan) → Tester: @yuliyang
• Chinese (Hong Kong) → Tester: @yuliyang
• Japanese → Tester: @nukaga & @miminari
• Korean → Tester: @donpark
• Vietnamese → Tester: @philiparthurmoore

[andersnoren]

@nielslange Updated!

[nielslange]

Thanks, @andersnoren!

[donpark]

@donpark Would it be an option to add Noto Sans KR using @font-face?

Yes! "Noto Sans KR" looks remarkably good. Here is a partial view (starring ㄱ,ㄴ, and ㄷ) of a collection of Hangul characters, including those that often breaks, aka 뷁어, using the font:

breaking-bad-kr.html.zip

Disclaimer: I'm just a casual user of Korean language who happens to be a developer, not an expert on Hangul nor Korean language.

UPDATE: Screenshot above is not a stereogram so you won't be able to see the hidden spaceship no matter how hard you squint.

[nielslange]

For Japanese, I think the title (h1-h6) choose to be Noto Sans JP and the text use be "Hiragino Kaku Gothic ProN" or "sans_serif" only. Web fonts in Japanese fonts are heavy and are not suitable for setting all posts. And since the environment of Japanese fonts is various, I thought it would be good to make use of the fonts of each device.

@nukaga You suggested using Hiragino Kaku Gothic ProN for the regular texts. It seems that this is a paid [premium font. Is that correct? If so, would you mind suggesting an alternative font for the regular texts from https://fonts.google.com/?subset=japanese?

[nielslange]

Yes! "Noto Sans KR" looks remarkably good. Here is a partial view (starring ㄱ,ㄴ, and ㄷ) of a collection of Hangul characters, including those that often breaks, aka 뷁어, using the font:

@donpark Thanks for confirming that Noto Sans KR would be a great match!

Disclaimer: I'm just a casual user of Korean language who happens to be a developer, not an expert on Hangul nor Korean language.

I'll reach out to more Korean folks to test the implementation of the Korean fallback font.

UPDATE: Screenshot above is not a stereogram so you won't be able to see the hidden spaceship no matter how hard you squint.

Damn it! I was trying to hard to see the spaceship. I should have read your entire message first! 😂

[nukaga]

@nielslange I'm sorry. My writing was not good. I thought it would be good to specify only sans_serif for regular text (use the default Gothic font for each device).

Because some people set “the default font easy to read”, and the fonts are set for each device for ease of viewing.

If we need to set the font, I think "Hiragino Kaku Gothic ProN" and "Meiryo" are good. Hiragino is a bundled font Mac OS X later. Meiryo has been bundled Windows vista later.

What do you think? @miminari

[nielslange]

@nukaga Don't worry about your writing and thanks for your quick reply! 😀 If I understand you correctly, you would use Noto Sans JP for the title and Hiragino Kaku Gothic ProN and Meiryo for the regular texts, correct?

[nukaga]

@nielslange

you would use Noto Sans JP for the title and Hiragino Kaku Gothic ProN and Meiryo for the regular texts, correct?

Yes, Thanks!

[nielslange]

Thank you, @nukaga! 🙏

[nielslange]

@alssong, @jsnmoon & @taggon Would you mind assisting me in testing fallback fonts for Korean?

[nielslange]

@dinhtungdu & @philiparthurmoore According to https://fonts.google.com/?subset=vietnamese Noto Sans supports Vietnamese. Do you have any objections using this font?

[dinhtungdu]

@nielslange No. Noto Sans is great for Vietnamese too :)

[nielslange]

Thanks, @dinhtungdu!

[nielslange]

@andersnoren Can you update the table in #118 (comment) as follows:

• Japanese → Fonts: Noto Sans JP, Hiragino Kaku Gothic ProN, Meiryo
• Korean → Font: Noto Sans KR
• Vietnamese → Font: Noto Sans
• Vietnamese → Tester: @dinhtungdu

[yuliyang]

@yuliyang Thanks, updated! Do you have any suggestions on suitable fallbacks for Noto Sans for the Chinese fonts? The following were used for Twenty Nineteen:

'PingFang TC', 'Helvetica Neue', Helvetica, STHeitiTC-Light, Arial, sans-serif;

Different languages will have different fallbacks.

Chinese Simplified (China)

'PingFang SC', 'Helvetica Neue', "Microsoft YaHei New", STHeiti Light, sans-serif

Chinese Traditional (Taiwan)

'PingFang TC', 'Helvetica Neue', "Microsoft YaHei New", STHeiti Light, sans-serif;

Chinese Hong Kong

'PingFang HK', 'Helvetica Neue', "Microsoft YaHei New", STHeiti Light, sans-serif;

Please correct me if anything doesn't look right. I'm not sure if different language devices have different default fonts. I'm using MacOS English system, not native Chinese system. I know 'PingFang xx' font for sure but not 100% sure anything after that.

[andersnoren]

@nielslange Updated! @yuliyang Awesome, big thanks for the help! Updated.

[nielslange]

Thanks, @yuliyang and @andersnoren! 🙏

[elazzabi]

@andersnoren @nielslange the Cairo font looks good for Arabic. I played with some sentences on the Google Fonts site and it looks good.

Any instructions on how to use the theme on a site so I can look around and see if everything looks good?

[nielslange]

@elazzabi Thanks for confirming the font. Can you tell me if it's common in Arabic to format texts bold and italic? Regarding testing, you can download the Twenty Twenty theme, including the non-latin font fallback, via https://github.com/nielslange/twentytwenty/archive/try/%23118.zip and install it on a test site. Once you've done that, you need to change the site language to Arabic.

@andersnoren Can you update the table in #118 (comment) as follows:

• Arabic → Font: Cairo