[Bug]: charset issues on Chinese Windows machines

[User670]

Version

v1.0.0

Operating System

Windows 10

Problem Description

I'm on a Chinese Windows laptop, and my audio devices have names that have Chinese characters in them. The game doesn't display them correctly.

Steps to Reproduce Problem

0. Make an audio device have a name that contain Chinese characters (maybe also dependent on system charset?)
1. Go to settings, as shown in the screenshot above

Relevant log output

No response

(Optional) Other useful information

No response

[chrislo27]

I can potentially solve this by changing what font is displayed for the device name. I don't know for certain if it is a charset issue since I would expect no glyphs to show up rather than what you've screenshotted.

Would you mind replying with a list of your audio device names?

[chrislo27]

Additionally, what is your OS's charset? You can check this via PowerShell by running [System.Text.Encoding]::Default. Source

[User670]

I can potentially solve this by changing what font is displayed for the device name. I don't know for certain if it is a charset issue since I would expect no glyphs to show up rather than what you've screenshotted.

Would you mind replying with a list of your audio device names?

I'm positive it's a charset issue. The weird characters shown is an example of mojibake.

I don't know what's supposed to be in the screenshot (I don't have an audio device with that name, probably it was something like "default"?). Another garbled text, shown in the screenshot below, is supposed to read 扬声器.

Running that command in powershell gives this:

BodyName          : gb2312
EncodingName      : 简体中文(GB2312)
HeaderName        : gb2312
WebName           : gb2312
WindowsCodePage   : 936
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
IsSingleByte      : False
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : True
CodePage          : 936

I did some quick test in Python, and the in-game garbled text seems to be GB2312 text being decoded with some version of ISO-8859.

[User670]

The screenshot in the original post... it has 11 bytes, the first 10 read "主声音驱动" in GB2312 (means "main audio drive(r)")

[chrislo27]

You're right that it is a charset issue, thank you for checking! I did some more digging and at a glance this appears to be a JDK issue: JDK-8177951

Supposedly this is fixed in JDK 10. Did you download the win64 version of the game or the platform_agnostic version? If you're able to, it would be helpful to test the platform_agnostic version using JDK 11. You can download prebuilt JDKs from AdoptOpenJDK.

If you're not able to set that up, I can try packaging it into the win64 version but I might need a day to sort that out.

[User670]

I'm on the win64 version. I can try the other version; my system's default Java is Java 15.

[chrislo27]

Yes, trying on the platform_agnostic version would be great, Java 15 should work fine too. Thanks!

[User670]

With Java 15, it refuses to display the Chinese characters, not sure if it's the font problem this time

[User670]

Unzipped your jar and double clicked the fonts one by one. In the font preview window, these three fonts had Chinese characters falling back to something else, implying that they don't support Chinese characters.

example:

The other three fonts do support Chinese characters, though I'm not sure which variety (China Mainland, Taiwan, Japan, Korea) the characters are. and they are all using the Japanese variant of the characters.

[chrislo27]

Yep, seems like the mojibake is gone but the Roboto font doesn't have the Chinese character glyphs as you found out.

Below is a download link to a zip file of a test build (v1.0.1) but using the Rodin font instead for the mixer label. GitHub annoyingly only allows 25 MB generic file uploads so I opted to upload it to Discord. Could you please test it and see if the Chinese characters are rendered? If this works then this + using JRE 10 or newer is the solution, otherwise I'll have to find a different font.

https://cdn.discordapp.com/attachments/306231796369195020/880636838392516668/prmania_platformagnostic_issue_4_a.zip

I also want to briefly mention that having better fallback font support is currently in discussions as part of localization efforts (https://github.com/chrislo27/PolyrhythmMania/discussions/3).

[User670]

Missing character duh, does Japanese not have the character 扬?

[chrislo27]

It seems Japanese doesn't use 扬 given that a hasty Google Translate from Japanese to English produces the same character.

Well, bummer. But at least we know that the charset issue is resolved and now it is just a font issue.

I will work on resolving this as best I can for a future patch update, likely using something like Noto Sans just for this one option's text before I work on the font fallback system. However, I greatly appreciate your help with debugging this so far!

Please leave this issue open and I'll update from time to time on progress.