UltraStar-Deluxe / USDX

The free and open source karaoke singing game UltraStar Deluxe, inspired by Sony SingStar™
https://usdx.eu
GNU General Public License v2.0
863 stars 162 forks source link

Songs with ä ö ü are missing in the 2023 version #669

Open LoZe opened 1 year ago

LoZe commented 1 year ago

Songs with German umlauts (ä,ü,ö) are no longer displayed in the 2023 version. it was shown to me in the 2020 version.

Raspberry Pi OS with desktop System: 64-bit Kernel version: 5.15

LoZe commented 2 days ago

I had thought that the problem was due to my error port compiling, but I also have it with the Windows.exe in the 2024.10 Version. What I have discovered so far, that the letters are converted incorrectly in the error log "Can't find audio file in song: ...".

it is ä = "61 CC 88" instead of ä = "C3 A4"

bohning commented 2 days ago

Seems to be a Unicode normalization issue, see e.g. https://minaret.info/test/normalize.msp

s09bQ5 commented 2 days ago

@LoZe, can you check with a hex editor which bytes are used in the .txt file of that song to represent the ä?

barbeque-squared commented 20 hours ago

It's probably something similar to #904, although that did fix special characters in at least the txt filename for me.

Just to make sure: this is with a special character in the #MP3 tag inside the txt? We can probably use the weird Č again because if this has the same cause as the other issue, that one appears to trigger it on most locales.

s09bQ5 commented 19 hours ago

As far as I understood @LoZe has this problem both on Linux and on Windows. So it is unlikely that it is an issue with the locale.

LoZe commented 11 hours ago

I think the problem occurred while moving the folders. I have now found out that the wrong character was contained in the file name and folder, the contents of the text file were correct. I wrote a small program that renames all files with these chars to normal chars. I can't test it at the moment but I'll let you know as soon as I can start Ultrastar.

bohning commented 4 hours ago

I wrote a small program that renames all files with these chars to normal chars.

But why would you want to do that?

LoZe commented 4 hours ago

because when I look for a band like "Die Ärzte", The Ä key on the keyboard should be the Ä in the name.