The demo snap partially renders text as squares.

[brlin-tw]

Refer the following screenshot:

Locale settings:

$ locale
LANG=zh_TW.UTF-8
LANGUAGE=zh_TW:zh_HK:zh_CN:en
LC_CTYPE="zh_TW.UTF-8"
LC_NUMERIC=zh_TW.UTF-8
LC_TIME=zh_TW.UTF-8
LC_COLLATE="zh_TW.UTF-8"
LC_MONETARY=zh_TW.UTF-8
LC_MESSAGES="zh_TW.UTF-8"
LC_PAPER=zh_TW.UTF-8
LC_NAME=zh_TW.UTF-8
LC_ADDRESS=zh_TW.UTF-8
LC_TELEPHONE=zh_TW.UTF-8
LC_MEASUREMENT=zh_TW.UTF-8
LC_IDENTIFICATION=zh_TW.UTF-8
LC_ALL=

Not sure if it is related to this project, though, as it is built on 2020/7/30 which may not reflect the current implementation.

[merlijn-sebrechts]

Thanks for testing this out, @Lin-Buo-Ren !

In order to reproduce this, how to I set those locales? I don't have a lot of experience with this.

[merlijn-sebrechts]

When I set those environment variables and run notepad-plus-plus, I get this result:

Can you test the snaps notepad-plus-plus and photoscape? They use the latest version of sommelier-core. This script is actually using the same fix created by Taqi when you told him about the issue in notepad-plus. But since I don't know how to switch locale, I haven't tested it myself.

[brlin-tw]

notepad-plus-plus:

photoscape:

[merlijn-sebrechts]

Can you run the following commands and show the output?

fc-match sans
find /usr/share/fonts/ -type f \( -name "*.ttf" -o -name "*.ttc" \)

And which OS and version are you running?

[brlin-tw]

fc-match sans

$ fc-match sans
NotoSansCJK-Regular.ttc: "Noto Sans CJK TC" "Regular"

find /usr/share/fonts/ -type f ( -name ".ttf" -o -name ".ttc" )

find-usr-share-fonts-type-f-name-ttf-ttc.out.txt

And which OS and version are you running?

I'm using Ubuntu 21.04.

[brlin-tw]

I am able to rule out the square symbols in the Wine initialization message by dropping a custom override in Fontconfig:

<?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
 <!-- Prefer Powerline fonts for console -->
 <match target="pattern">
  <test qual="any" name="family">
   <string>monospace</string>
  </test>
  <edit binding="strong" mode="prepend" name="family">
   <string>DejaVu Sans Mono</string>
   <string>Noto Sans Mono CJK TC</string>
   <string>WenQuanYi Zen Hei Mono</string>
   <string>PowerlineSymbols</string>
   <string>HYSong</string>
   <string>DejaVu Sans Mono</string>
   <string>Bitstream Vera Sans Mono</string>
  </edit>
 </match>
 <dir>~/.fonts</dir>
</fontconfig>

However the snapped application still renders text as square symbols regardless.

[merlijn-sebrechts]

Aha, that's good to know!

So, the issue is that, for some reason, the apps default to a font which doesn't have Latin characters. I'm not sure why the apps would use a different font than the wine initialization dialog, though.

I've tried to replicate a zh_TW.UTF-8 environment but there, everything works like it should (using Noto Sans CJK TC everywhere).

[merlijn-sebrechts]

@mmtrt I think I need your help here. Do you have any idea what might be going wrong here?

[mmtrt]

Could be font cache issue since he has some modified font config so rebuild font cache.

OR

he should check wine registry for font substitute change have applied correctly.

Also Since he has Windows fonts installed in system could create conflict with wine font changes not sure though.

[brlin-tw]

Could be font cache issue since he has some modified font config so rebuild font cache.

I've rebuilt the font cache, issue still can be reproduced.

he should check wine registry for font substitute change have applied correctly.

Any guidance?

Also Since he has Windows fonts installed in system could create conflict with wine font changes not sure though.

I'll check it out.

[mmtrt]

Open regedit with notepad-plus-plus.wine regedit then browse HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\FontSubstitutes and post all info of this here.

EDIT: also post of this too HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontLink\SystemLink

[brlin-tw]

Open regedit with notepad-plus-plus.wine regedit then browse HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\FontSubstitutes and post all info of this here.

[HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\FontSubstitutes]
"Arial Baltic,186"="Arial,186"
"Arial CE,238"="Arial,238"
"Arial CYR,204"="Arial,204"
"Arial Greek,161"="Arial,161"
"Arial TUR,162"="Arial,162"
"Courier New Baltic,186"="Courier New,186"
"Courier New CE,238"="Courier New,238"
"Courier New CYR,204"="Courier New,204"
"Courier New Greek,161"="Courier New,161"
"Courier New TUR,162"="Courier New,162"
"Helv"="MS Sans Serif"
"Helvetica"="Arial"
"MS Shell Dlg"="Tahoma"
"MS Shell Dlg 2"="Tahoma"
"Times"="Times New Roman"
"Times New Roman Baltic,186"="Times New Roman,186"
"Times New Roman CE,238"="Times New Roman,238"
"Times New Roman CYR,204"="Times New Roman,204"
"Times New Roman Greek,161"="Times New Roman,161"
"Times New Roman TUR,162"="Times New Roman,162"
"Tms Rmn"="Times New Roman"

EDIT: also post of this too HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontLink\SystemLink

[mmtrt]

@Lin-Buo-Ren looks like snap didn't add registry changes for cjk font that's why its not working.

[mmtrt]

@Lin-Buo-Ren can you test npp from edge see if new changes do anything for you.

aahh npp app install fail on snap will fix it later but anyways you can check winecfg and regedit these should be in your locale now.

notepad-plus-plus.wine regedit

notepad-plus-plus.wine winecfg

[brlin-tw]

npp revision 264:

[HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\FontSubstitutes]
"Arial Baltic,186"="Arial,186"
"Arial CE,238"="Arial,238"
"Arial CYR,204"="Arial,204"
"Arial Greek,161"="Arial,161"
"Arial TUR,162"="Arial,162"
"Courier New Baltic,186"="Courier New,186"
"Courier New CE,238"="Courier New,238"
"Courier New CYR,204"="Courier New,204"
"Courier New Greek,161"="Courier New,161"
"Courier New TUR,162"="Courier New,162"
"Helv"="MS Sans Serif"
"Helvetica"="Arial"
"MS Shell Dlg"="Noto Sans CJK TC"
"MS Shell Dlg 2"="Tahoma"
"Times"="Times New Roman"
"Times New Roman Baltic,186"="Times New Roman,186"
"Times New Roman CE,238"="Times New Roman,238"
"Times New Roman CYR,204"="Times New Roman,204"
"Times New Roman Greek,161"="Times New Roman,161"
"Times New Roman TUR,162"="Times New Roman,162"
"Tms Rmn"="Noto Sans CJK TC"

[merlijn-sebrechts]

@Lin-Buo-Ren Did you see the same squares on the wine initialization dialog?

[brlin-tw]

@Lin-Buo-Ren Did you see the same squares on the wine initialization dialog?

Yes:

[merlijn-sebrechts]

@Lin-Buo-Ren Now that I think of it, that dialog is shown before the registry changes are made. First we initialize wine and then we run regedit to create custom overrides for cjk fonts. The Wine environment needs to exist before we can run regedit. Can you open up notepad-plus-plus.wine winecfg and see if that has the same issue?

You said earlier:

I am able to rule out the square symbols in the Wine initialization message by dropping a custom override in Fontconfig:

Given that the Wine initialization message runs before we apply our font overrides, this means that the default font Wine uses works correctly when you remove your custom overrides in .fontconfig. :thinking:

[brlin-tw]

Can you open up notepad-plus-plus.wine winecfg and see if that has the same issue?

I removed the ~/snap/notepad-plus-plus directory before launching winecfg

this means that the default font Wine uses works correctly when you remove your custom overrides in .fontconfig. thinking

Not sure, note that I've dropped the custom override as it doesn't actually solve anything.

[brlin-tw]

For revision 266 of notepad-plus-plus:

Wine initialization window is proper:

The main interface also proper, except for the non-localized text which should be a different issue:

The interface text mostly rendered properly after switching the language to Chinese, except for the "Close" button:

Comparison:

However, some text in the language selection drop-down rendered as crossed rectangular, which should be rather harmless, though...

Another issue is that by default Chinese text input is rendered as squares:

(notice the pre-commit character selection UI displays out of the window, which should be a separate issue (in Wine))

I can workaround the issue by changing the default editor font in the settings:

Do font fallbacks supported by Wine even?

[merlijn-sebrechts]

@Lin-Buo-Ren I'm reviewing https://github.com/snapcrafters/sommelier-core/pull/13 but I need your guidance since I can't actually read this language.

I'm using these locales:

export LANG=zh_TW.UTF-8
export LANGUAGE=zh_TW:zh_HK:zh_CN:en
export LC_CTYPE="zh_TW.UTF-8"
export LC_NUMERIC=zh_TW.UTF-8
export LC_TIME=zh_TW.UTF-8
export LC_COLLATE="zh_TW.UTF-8"
export LC_MONETARY=zh_TW.UTF-8
export LC_MESSAGES="zh_TW.UTF-8"
export LC_PAPER=zh_TW.UTF-8
export LC_NAME=zh_TW.UTF-8
export LC_ADDRESS=zh_TW.UTF-8
export LC_TELEPHONE=zh_TW.UTF-8
export LC_MEASUREMENT=zh_TW.UTF-8
export LC_IDENTIFICATION=zh_TW.UTF-8
export LC_ALL=

1. This is winecfg before that PR:
2. This is winecfg after that PR:

What surprised me is that some of the symbols are quite different between the two versions. What is the difference between the symbols and which version is correct for that locale? Or are these the same symbols, just in a different font?

[mmtrt]

I've resolved that issue locally I'll push changes soon.

[brlin-tw]

What surprised me is that some of the symbols are quite different between the two versions. What is the difference between the symbols and which version is correct for that locale?

The previous version is using serif-like 楷(Kai) style font, the latter one uses san-serif like 黑(Hei) style font, which is more proper considering the medium is the screen.

[mmtrt]

@Lin-Buo-Ren @galgalesh Test this snap thoroughly for any issues regarding cjk locales so that I can push my changes to PR thanks. notepad-plus-plus_267.tar.gz

see below for snap download link.

[merlijn-sebrechts]

@mmtrt

I still have the same issue with the rectangles.

I did a bunch of investigating yesterday and I figured out why:

• The modern theme uses the ubuntu font face.
• The ubuntu font is present on my system.

  ls ~/snap/notepad-plus-plus/common/.wine/drive_c/windows/Fonts/ | grep Ubuntu
  Ubuntu-BI.ttf
  Ubuntu-B.ttf
  ...

For this reason, Wine uses Ubuntu instead of the Noto font in a bunch of places. Since the Ubuntu font doesn't contain CJK characters, this shows as rectangles. If I remove the Ubuntu fonts, the UI falls back to the Noto font.

Regardless of the theme, this will happen in other apps too. Any program explicitly specifying a font will break in the following conditions:

• The font is installed and available as ttf or ttc
• The font doesn't have CJK symbols

I have this issue with the PhotoScape UI, regardless of which Theme I use because Photoscape explicitly specifies another font.

Ways to fix this:

0: Switch to a theme which uses regular MS fonts

If a theme uses the (pseudo)fonts "MS Shell Dlg" and "Tms Rmn", that will fix the issue for all Wine UI. The default Wine theme uses those fonts.

This approach isn't complete, however, because apps like PhotoScape specify a specific font for the UI which isn't affected by the theme.

1: Get font fallback working.

Wine supports very rudimentary font fallback. One of the issues is that this doesn't work automatically on all text: "Fallback fonts will only be used in programs and areas where the ScriptString apis are used" I tried to configure it, but this doesn't seem to fix the issue. This is what I added to the registry (copied from user.reg):

  [Software\\Wine\\Uniscribe\\Fallback] 1623356681
  #time=1d75e36a4c15e00
  "0x696e6168"="Noto Sans CJK TC"

I don't know whether I made a mistake in the configuration or whether the font fallback doesn't work for buttons.

2: Override all fonts possibly used in the UI to Noto Sans

This is what they use here: https://gist.github.com/swordfeng/c3fd6b6fcf6dc7d7fa8a

I haven't tried this, but I expect it to work. This has the downside that those fonts will always be substituted, even in, for example, Microsoft Word.

[mmtrt]

@galgalesh did you tried on fresh snap prefix without any snap data?

It should have worked since I replaced font fallback to noto cjk which font changed after using LANG env

Thing is wine resets font fallback when user changes LANG env so registry changes are lost in predefined registry entries which could be cause of button font as boxes. So user have to use that LANG for it to not reset.

Also check for font substitutes when changing LANG env "MS Shell Dlg" and "Tms Rmn" will font names will changes when switching LANG then we have to change these fonts fallback to noto cjk from systemlink which I did currently though it might be change in your system.

[merlijn-sebrechts]

did you tried on fresh snap prefix without any snap data?

I ran snap remove --purge notepad-plus-plus before installing it and trying it out. I just tried it again, and I have the same result.

How did you change font fallback? Then I can check if it's set on my system.

I can't see any uniscribe fallback config in the registry:

cat ~/snap/notepad-plus-plus/common/.wine/*.reg | grep -i fallback

Also check for font substitutes when changing LANG env "MS Shell Dlg" and "Tms Rmn" will font names will changes when switching LANG then we have to change these fonts fallback to noto cjk from systemlink which I did currently though it might be change in your system.

I don't completely understand what you're saying here. This is what I do to test:

$ export LANG=zh_TW.UTF-8
export LANGUAGE=zh_TW:zh_HK:zh_CN:en
export LC_CTYPE="zh_TW.UTF-8"
export LC_NUMERIC=zh_TW.UTF-8
export LC_TIME=zh_TW.UTF-8
export LC_COLLATE="zh_TW.UTF-8"
export LC_MONETARY=zh_TW.UTF-8
export LC_MESSAGES="zh_TW.UTF-8"
export LC_PAPER=zh_TW.UTF-8
export LC_NAME=zh_TW.UTF-8
export LC_ADDRESS=zh_TW.UTF-8
export LC_TELEPHONE=zh_TW.UTF-8
export LC_MEASUREMENT=zh_TW.UTF-8
export LC_IDENTIFICATION=zh_TW.UTF-8
export LC_ALL=
merlijn@howard:~/Downloads$ sudo snap remove --purge notepad-plus-plus 
[sudo] merlijn 的密碼： 
notepad-plus-plus 已被移除
merlijn@howard:~/Downloads$ sudo snap install notepad-plus-plus_267.snap --dangerous
notepad-plus-plus 8.0 installed
merlijn@howard:~/Downloads$ notepad-plus-plus

[mmtrt]

I can't see any uniscribe fallback config in the registry:

fallback fonts are in here HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\FontLink\SystemLink

"MS Shell Dlg" and "Tms Rmn"

I mean when user uses cjk locale wine changes fonts from fallback according to cjk region that uses it Wiki List of cjk fonts

I ran snap remove --purge notepad-plus-plus before installing it and trying it out. I just tried it again, and I have the same result.

Sorry I mistakenly uploaded wrong rev that I downloaded from edge didn't had any local changes so this should do it.

notepad-plus-plus_8.0_amd64.tar.gz

[merlijn-sebrechts]

This works great!

Amazing work, @mmtrt!

I didn't know about fontlinks. This is apparently a second method for font fallback.

Unlike font fallback, in which the selected font is internally replaced by a predefined font, in font linking (also called “composite fonts”) it is possible to link one or more fonts (called "linked fonts") to another font (called the "base font"). Once you link fonts, you can use the base font to display code points that do not exist in the base font, but that do exist in one of the linked fonts. For example, linking a Hangeul font and a Japanese font to a Tahoma font allows you to display both Korean and Japanese characters while specifying Tahoma font. The Noto family of fonts uses the model to create what appears as a single font by combing script-specific font files.

source

There is one thing I do not understand, though. Modern theme uses Ubuntu font for buttons. There is not font link defined for Ubuntu font. Then why does it still fall back to Noto with this configuration?

[mmtrt]

There is one thing I do not understand, though. Modern theme uses Ubuntu font for buttons. There is not font link defined for Ubuntu font. Then why does it still fall back to Noto with this configuration?

Since wine doesn't use ubuntu font by default anywhere so it ignores theme config I guess and ReactOS uses it by default in system.

So with this new cjk config we are using noto font in predefined fallback fonts and also adding fonts of fallback fonts from predefined data such as Batang, Gulim, SimSun, NSimSun, MingLiU, PMingLiU, MS Serif, MS UI Gothic entries separately then adding noto font as fallback for these cjk region fonts as per cjk config.

[merlijn-sebrechts]

Changes are merged into master and 1.0 branches. Should be fixed in the apps the next time they get rebuild.