Regression in macOS Sequoia

[NathanCummings]

What Operating System(s) are you seeing this problem on?

macOS

Which Wayland compositor or X11 Window manager(s) are you using?

No response

WezTerm version

20240928-215536-a2f2c07a

Did you try the latest nightly build to see if the issue is better (or worse!) than your current version?

Yes, and I updated the version box above to show the version of the nightly that I tried

Describe the bug

EDIT: 19th October 2024 This is most easily reproduced by using the CharSelect tool (Ctrl+Shift+U) and choosing a half-width katakana character (e.g., FF6B) and pressing enter.

Since I updated to Sequoia, the "neo" program (https://github.com/st3w/neo) hasn't been working properly. The glyphs aren't showing and it is very laggy. I wasn't sure where to report this until I noticed that it is working as expected in both kitty and the default Terminal app on macOS, as can be seen in the image below.

To Reproduce

Installing and running neo on a Mac with macOS Sequoia is how I produce the problem. I have another machine still on macOS Sonoma and it works fine. I checked with no config and the issue was still present.

Configuration

no config

Expected Behavior

See the image with kitty (middle) and Terminal (right) to see what it should look like.

Logs

No response

Anything else?

No response

[NathanCummings]

Debug Overlay
wezterm version: 20240928-215536-a2f2c07a aarch64-apple-darwin
Window Environment: macOS 15.0.1 (24A348)
Lua Version: Lua 5.4
OpenGL: Apple M3 4.1 Metal - 89.3
Enter lua statements or expressions and hit Enter.
Press ESC or CTRL-D to exit
10:05:24.010 WARN wezterm_font > No fonts contain glyphs for these codepoints: \u{ff91}.
Placeholder glyphs are being displayed instead.
You may wish to install additional fonts, or adjust your
configuration so that it can find them.
https://wezfurlong.org/wezterm/config/fonts.html has more information about configuring fonts.
Set warn_about_missing_glyphs=false to suppress this message.
>

I'm not sure if this helps, since other terminal emulators are not failing to find the glyphs, but I'll put it here anyway.

[NathanCummings]

In fact, running wezterm ls-fonts --list-system shows that Wezterm is struggling with many oriental characters. The Neo application is just an example of this.

I've checked with different fonts as well as the default.

[hamano]

I believe this is simply due to missing font characters. It is unrelated to the EAW issue. neo uses half-width katakana (U+FF71 and beyond, EAW: halfwidth), so you need to use a font that covers this. The fonts you listed do not appear to be japanese font.

I'm not familiar with macOS, but I believe the default installed japanese font is 'Hiragino'. Hiragino is not very suitable for terminal use, so it might be a good idea to install a different font.

In my environment (Linux), where I have installed Japanese fonts, Neo runs without any issues on wezterm. The font I am using is this one: https://github.com/hamano/font-eaw

[NathanCummings]

I believe this is simply due to missing font characters. It is unrelated to the EAW issue. neo uses half-width katakana (U+FF71 and beyond, EAW: halfwidth), so you need to use a font that covers this. The fonts you listed do not appear to be japanese font.

I'm not familiar with macOS, but I believe the default installed japanese font is 'Hiragino'. Hiragino is not very suitable for terminal use, so it might be a good idea to install a different font.

In my environment (Linux), where I have installed Japanese fonts, Neo runs without any issues on wezterm. The font I am using is this one: https://github.com/hamano/font-eaw

The thing that is confusing me is that other terminal emulators are managing ok, I don't know how to debug this one.

That, and the fact that it was upgrading to the new macOS that broke this.

[hamano]

If fonts are not explicitly specified, it's possible that OS updates could cause changes in behavior.

You can easily debug this by doing the following. I tested it on Sonoma since my Mac cannot be updated to the latest version.

$ ./wezterm ls-fonts --codepoints 3042
LeftToRight
 0 あ    \u{3042}     x_adv=10 cells=2  glyph=852  wezterm.font(".Apple SD Gothic NeoI", {weight="Regular", stretch="Normal", style="Normal"})
                                      /System/Library/Fonts/AppleSDGothicNeo.ttc index=1 variation=0, CoreText

$ ./wezterm ls-fonts --codepoints FF71
LeftToRight
 0 ｱ    \u{ff71}     x_adv=6  cells=1  glyph=42591 wezterm.font(".PingFang HK", {weight="Regular", stretch="Normal", style="Normal"})
                                      /System/Library/Fonts/PingFang.ttc index=18 variation=0, CoreText

U+3042 (あ) is a common Japanese character. AppleSDGothicNeo was selected. U+FF71 (ｱ) is a less frequently used, minor character in Japanese. A Chinese font called PingFang was selected.

I'm not sure how things work on the latest macOS, but to resolve this issue, you should explicitly set the font.

[hamano]

'.Hiragino Kaku Gothic Interface' seems appropriate, but it was unavailable I don't know why... Instead, you can use the proportional font 'Hiragino Kaku Gothic Pro'.

$ cat ~/.wezterm.lua                                                                     
local wezterm = require 'wezterm'
local config = wezterm.config_builder()
config.font = wezterm.font_with_fallback {
  'Hiragino Kaku Gothic Pro',
}
return config

The better solution would be to install the font from the following URL: https://github.com/hamano/font-eaw/releases

config.font = wezterm.font_with_fallback {
  'EAW CONSOLE',
}

[NathanCummings]

OK, so I ran wezterm ls-fonts --codepoints FF71 on two Macs, one running Sonoma and one running Sequoia.

Sonoma:

Sequoia:

If fonts are not explicitly specified, it's possible that OS updates could cause changes in behavior.

This certainly seems to be what is happening here.