search

ComunidadAylas / PackSquash

📦 Minecraft: Java Edition resource and data pack optimizer which aims to achieve the best possible compression, performance and protection, improving pack distribution, storage and in-game load times.
https://packsquash.aylas.org
GNU Affero General Public License v3.0
615 stars 25 forks source link

Special characters in German language translation are considered as surrogates and halt compression #340

Open qdpy opened 1 week ago

qdpy commented 1 week ago

Distribution

N/A (the distribution does not matter; please explain why if it's not obvious)

Bug description

The de_de.json translation file contains special characters that are considered as lone leading surrogates, which leads to JSON parsing errors and stops compression. The error message is as follows:

! assets/minecraft/lang/de_de.json: JSON error: lone leading surrogate in hex escape at line 1 column 22445

"F\u00fcnfbl\u00e4ttriges Kleeblatt"

Reproduction steps

Compress the pack

Expected behavior

-

Attached files

No response

Additional context

No response

AlexTMjugador commented 1 week ago

PackSquash supports Latin letters with diaeresis in their escaped form, along with most other characters from the Basic Multilingual Plane (BMP), as expected. I verified this by creating a pack file at assets/minecraft/lang/de_de.json with the following content: "F\u00fcnfbl\u00e4ttriges Kleeblatt". PackSquash processed the file successfully.

If you can reproduce this same result on your end, which I consider extremely likely, the issue cannot stem from these specific characters. Moreover, the high column count in the error message suggests that the file is a minified, non-vanilla JSON, which can make pinpointing the exact error location and cause more difficult, so could you attach the problematic file for analysis? I don't see this issue being investigated further without clear evidence of its source.