Different URL parsed on Windows and Linux when non-ASCII chars are present

Urabartin commented 1 month ago

Steps to reproduce

Run Komga on Windows, scan a library with some non-ASCII characters in path
Run Komga in Docker on Manjaro, with the same config directory and data directory, then scan the library again

Expected behavior

The old series can be recognized as existed and not overrided, with all reading progresses preserved.

Actual behavior

Book progresses are preserved, until a scan on the library is run. The old series are marked as 'unavailable' and replaced by new series with the same name by different url (in sqlite). For example, a series with url E:\test\心 is replaced by one with E:\test\%E5%BF%83.

Logs

Logged on Windows:

2024-08-13T11:00:07.045+08:00  INFO 4328 --- [taskProcessor-19] o.g.k.d.service.LibraryContentLifecycle  : Adding new series: Series(name=心ぴーち, url=file:/E:/.hide/test/心ぴーち/, fileLastModified=2024-08-11T18:53:47.694035300, id=0GYPHPDB3YBF8, libraryId=0GYPHPAPQY94P, bookCount=0, deletedDate=null, oneshot=false, createdDate=2024-08-13T11:00:06.616193500, lastModifiedDate=2024-08-13T11:00:06.616193500)

Logger on Manjaro (Docker):

2024-08-13T18:19:58.401+08:00  INFO 1 --- [taskProcessor-2] o.g.k.d.service.LibraryContentLifecycle  : Adding new series: Series(name=心ぴーち, url=file:/E:/.hide/test/%E5%BF%83%E3%81%B4%E3%83%BC%E3%81%A1/, fileLastModified=2024-08-11T18:53:47.694035300, id=0GYSPC1XCJE8D, libraryId=0GYPHPAPQY94P, bookCount=0, deletedDate=null, oneshot=false, createdDate=2024-08-13T18:19:58.315059064, lastModifiedDate=2024-08-13T18:19:58.315059064)

Almost full log: pastebin

Komga version

1.11.2

Operating system

Windows 10 & Manjaro

Installation method

other (please provide more details below)

Other details

Installation method: Jar on windows (by Scoop); Docker compose on Manjaro

Honestly it is a wierd scenario attempting synchronize between Linux and Windows with a removable disk instead of something like NAS, but it would be helpful if it can be done.

Acknowledgements

[X] I have searched the existing issues (open AND closed) and this is a new ticket, NOT a duplicate or related to another open issue.
[X] I have written a short but informative title.
[X] I have checked the FAQ.
[X] I have updated the app to the latest version.
[X] I will fill out all of the requested information in this form.

gotson commented 4 weeks ago

probably your locale not setup properly on Linux

Urabartin commented 4 weeks ago

I have tried forcing the following environment variables as en_US.UTF-8 or ja_JP.UTF-8 in docker-compose.yaml, but it did not help.

    environment:
      - LANG=ja_JP.UTF-8
      - LC_ALL=ja_JP.UTF-8
      - LANGUAGE=ja_JP.UTF-8

gotson / komga