Text processing works under local codepage when it should use UTF-8

qixils / minecraft-randomiser

Resource pack and data pack randomiser to shuffle textures, sounds, loot tables, recipes, and more

MIT License

46 stars 6 forks source link

Text processing works under local codepage when it should use UTF-8 #5

Closed vitalline closed 5 years ago

vitalline commented 5 years ago

After opening UTF-8 translation files in local codepage (which in my case is cp65001), readlines() crashes when working with UTF-8. The same thing happens when writing UTF-8 using write() into a file opened in local codepage. This can be fixed by adding encoding='utf-8' to the open() functions (particularly on lines 196, 225 and 236 of resource_randomiser.py).

qixils commented 5 years ago

that should hopefully fix it, though this weirdly didn't show up in any of my testing (even my basic tests on windows)

vitalline commented 5 years ago

Yeah, that's just about what I did when trying to get it to work. It's probably a locale-specific problem, given that I use the Russian locale and encounter problems because of it relatively frequently.