search

iquercorb / OpenModMan

Open Mod Manager - Open source and generic Mod ("Modifications") manager.
GNU General Public License v3.0
88 stars 5 forks source link

UTF-8 Support #33

Closed gdinit closed 2 years ago

gdinit commented 2 years ago

Hi,

Thanks for this software! First time user, just downloaded the latest release ("0.9.9.5 (Beta - hotfix)") and tried to use it to mod-manage the following combo: mod = Epic Mod ( https://coldhunter.ru ), for game = Cold Waters ( https://store.steampowered.com/app/541210/cold_waters/ ).

If I am understanding this correctly, "Create Package" step failed when it tries to write a ZIP archive. This happened when I tried to create a package from extracted mod base directory.

The details of the issue with Cyrillic Characters: It occurred when ZIP'ing process tried to read the file "что это.txt". I tried to workaround the issue by renaming that particular file: FROM: Cold Waters Epic Mod 2.34 (21.01.22)\ColdWaters_Data\StreamingAssets\epicmod\audio\engine\что это.txt TO: Cold Waters Epic Mod 2.34 (21.01.22)\ColdWaters_Data\StreamingAssets\epicmod\audio\engine\RENAMED_FILE.TXT

Then, I retried the process and it failed on another file. I gave up on my workaround attempt and decided to create an issue and ask about UTF-8. Based on this experience, I would like to ask whether UTF-8 support is on the Roadmap? (I assume it would fix this issue?).

If this should be possible with the software and I am missing something please let me know (I did check the README and wiki for keywords like UTF, encoding etc. but couldn't find any mentions).

Thanks again!

Edit: I forgot to say, the way I found your software was via your older Mod Manager software OvGME
( https://forum.dcs.world/topic/144817-new-gme-software/ ). Having experienced this issue, I went back to OvGME and it worked fine. I get that OvGME is outdated, not developed, less featured but it did the job for me today. In the future I would like to switch to OpenModMan when it can handle Cyrillic characters.

Edit2: typo

iquercorb commented 2 years ago

Open Mod Manager normally DOES already support unicode with charset conversions. So, cyrillic characters SHOULD work fine everywhere but, indeed, the ZIP write/read module is where problems may occurs.

gdinit commented 2 years ago

I was already impressed with the vision (as outlined in the README) and the fact it is not only a design-doc but there is an implementation ready to test/use, that has been actively developed for almost 2 years now.

Great to see quick response & support as well.

Once this small zip-process encoding issue is fixed, I will be very keen to switch from OvGME to this software.

iquercorb commented 2 years ago

You tell me this work properly with OvGME ? Because I yes, I don't understand how...

iquercorb commented 2 years ago

So... the good news is that I identified the problem (this is the miniz.c library) and able to workaround. The bad news is that the workaround is ugly (to my standards), so I think I will take some time to "fork" and reimplement the miniz.c library with full and straight WinAPI file I/O and UTF-16 support.

gdinit commented 2 years ago

You tell me this work properly with OvGME ? Because I yes, I don't understand how...

Yes, that game (Cold Waters) and the mod I was trying (EpicMod, with Cyrillic characters) does work well with OvGME. OvGME does enable fine and it also does disable fine.

Edit3: ActualIy, I started to worry about a potential scenario "what if OvGME was quietly failing in bringing in files with Cyrillic characters" and I somehow did not notice it earlier... So I checked. Please see the screenshot, it appears the file I was unable to bring in using OpenModMan (due to miniz.c lib issue) are indeed brought in, using OvGME. This is an image

So... the good news is that I identified the problem (this is the miniz.c library) and able to workaround. The bad news is that the workaround is ugly (to my standards), so I think I will take some time to "fork" and reimplement the miniz.c library with full and straight WinAPI file I/O and UTF-16 support.

Glad to hear it! Thanks again and no rush on my part anyway, your other software is doing the job fine for now. I see that this is much improvement over OvGME so will upgrade to this in the future when I can.

Edit: added clarification that "OvGME can disable fine". Edit2: added game/mod names. Edit3: added the bit about double checking & the screenshot link.

iquercorb commented 2 years ago

Ok, I don't exactly know how and why this works on OvGME, I tryed to reproduce the OvGME implementation (pretty basic) and that doesn't worked.

ANYWAY, I managed to implement proper support in OMM in an elegant way and without rewritting miniz library. ZIP implementation now should fully support any file path with non-ANSI character.

This is not exactly an UTF-8 support since, in fact, the problem was never UTF-8 support (which work properly) but - as usual - Windows Specific charsets management, which, precisely, does not natively support UTF-8...

New version available here : https://github.com/sedenion/OpenModMan/releases/tag/0.9.9.6

iquercorb commented 2 years ago

I don't know if you noticed, but the 0.9.9.6 version did not correctly compress files. I juste released a new version with this bug fixed.