search

pkolaczk / fclones

Efficient Duplicate File Finder
MIT License
1.91k stars 71 forks source link

File not found (os error 2) when removing duplicates with non-Unicode symbols in the path name #102

Closed cela96 closed 2 years ago

cela96 commented 2 years ago

After fclones scanned and correctly identified the duplicate files I tried to run the command fclones link <dupes.txt on a folder mounted with samba on ubuntu 20.04-wsl and I got this error on files with very long names.

I received same error with fclones move duplicatefolder/ <dupes.txt

The base filesystem is ZFS on TrueNAS-12.0-U8

fclones:  warn: Failed to read metadata of /home/mount/longfoldername/youtube/Kaos/#134 Come temprare l'acciaio e qualche curiosit� (semplice tutorial)/#134 Come temprare l'acciaio e qualche curiosit� (semplice tutorial) (italiano_ASR).srt: No such file or directory (os error 2)
fclones:  warn: Could not determine files to drop in group with hash e85d86851b7fabba21fa20c426304b50 and len 8330: Metadata of some files could not be obtained

Could the problem be that the path is too long? Could it be related to the character à that is not recognized correctly? Could it be an ubuntu and samba problem?

Grazie Francesco

P.S. I ran this command in a cloned test dataset to avoid data loss of any kind so I'm not worried about any lost data.

th1000s commented 2 years ago

The os error 2 translates to ENOENT: Error: No such entry / entity, so fclones can't find the given file. If you can "ls" the file normally then this is probably caused by fclones running .to_string_lossy() to deal with invalid Unicode filenames. This call converts the 8-bit à to �. Working with raw 8-bit sequences everywhere would be possible but is rather tedious. Here the filename is probably encoded with ISO 8859-1 or -15, try converting it and others to proper Unicode, then it should work.