search

conicalflask / fs2

A sophisticated file sharing system for LAN parties with a focus on fast accurate searching, easy browsing and fast transfers.
BSD 3-Clause "New" or "Revised" License
6 stars 1 forks source link

Some filenames are not hashable #14

Open ghost opened 12 years ago

ghost commented 12 years ago

Upon attempting to hash some music, I've found that some filename characters cause it to fail:-

2012.08.08 19:46:25 WARNING: Failed to generate hash for 03 - H�n J�r�....mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/03 - H�n J�r�....mp3 (No such file or directory) 2012.08.08 19:46:25 WARNING: Failed to generate hash for 07 - Hafss�l.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/07 - Hafss�l.mp3 (No such file or directory) 2012.08.08 19:46:25 WARNING: Failed to generate hash for 06 - 18 Sek�ndur Fyrir S�laruppr�s.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/06 - 18 Sek�ndur Fyrir S�laruppr�s.mp3 (No such file or directory) 2012.08.08 19:46:25 WARNING: Failed to generate hash for 02 - D�gun.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/02 - D�gun.mp3 (No such file or directory) 2012.08.08 19:46:25 WARNING: Failed to generate hash for 01 - Sigur R�s.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/01 - Sigur R�s.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 08 - Ver�ld N� Og ��.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/08 - Ver�ld N� Og ��.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 04 - Leit A� L�fi.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/04 - Leit A� L�fi.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 11 - Syndir Gu�s (Opinberun Frelsarans).mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[1997] Von/11 - Syndir Gu�s (Opinberun Frelsarans).mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 06 - S�gl�pur.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/06 - S�gl�pur.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 10 - Svo hlj�tt.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/10 - Svo hlj�tt.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 04 - Me� bl��nasir.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/04 - Me� bl��nasir.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 07 - M�lan�.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/07 - M�lan�.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 03 - Hopp�polla.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/03 - Hopp�polla.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 02 - Gl�s�li.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/02 - Gl�s�li.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 11 - Heys�tan.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/11 - Heys�tan.mp3 (No such file or directory) 2012.08.08 19:46:26 WARNING: Failed to generate hash for 05 - S� lest.mp3, java.io.FileNotFoundException: /space/music/Sigur Rós/[2005] Takk/05 - S� lest.mp3 (No such file or directory)

I'm running on Ubuntu 11.04 , Java 6 OpenJDK

conicalflask commented 12 years ago

Yeah. This has been around forever that some files show up in listings but then appear to be unavailable to open due to weird chars in the filenames.

I think it's probably a manifestation of this "not a bug" JVM feature: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4733494 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4866151

The long and the short of it is these filenames contain unicode characters that don't exist in the locale that you ran Java in.

Try: LC_ALL=en_US java -jar fs2client-X.X.X.jar

I'll investigate if I can set the system locale at runtime or in the jar manifest but the claim is that using the system local for directory path strings is baked deep into Java and nobody considers it a bug.

conicalflask commented 12 years ago

Seems like I might be able to do: Locale.setDefault(newLocale);

If you report success with LC_ALL I'll have a play around with the code to set the default locale to en_US or en_UK. The C locale is a bit future resistant it seems.

mt-inside commented 12 years ago

Well, it is the "C" locale... FWIW, fsfuse checks the locale (though the posix nl_langinfo(CODESET) call) and refuses to start if it's not unicode. Better than loads of random bugs in shift-jis, I decided.

conicalflask commented 12 years ago

Assigned bigman while we wait for feedback about locales.

ghost commented 12 years ago

Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale.

conicalflask commented 12 years ago

Which locale? en_US is the one to try. I think I was wrong earlier and en_UK doesn't even exist. (it's en_GB)

ghost commented 12 years ago

That was the en_US locale.