pkolaczk / fclones

Efficient Duplicate File Finder
MIT License
1.87k stars 70 forks source link

fclones group --transform flag output not useable by fclones move #125

Closed peddamat closed 2 years ago

peddamat commented 2 years ago

Running the following command, (which compares RAW files using their embedded thumbnail):

fclones group --cache --transform 'dcraw -c -e $IN' --format json -o output.json /mnt/photos

results in fclones move throwing the following:

[2022-05-11 15:25:33.626] fclones:  warn: Skipping file '/mnt/h/Photo Archive/2020/2020-08-19/f51562240.raf' with length 56.1 MB different than the group length 2657671

As dcraw -c outputs to stdout, my thought is that fclones is saving the temp file's filesize, instead of the original file's filesize to the output json.

This causes a subsequent fclones move to throw the warning about the filesize mismatch, preventing deduplication.

pkolaczk commented 2 years ago

Thanks! I can see two things wrong here:

  1. as you noted, the size check should not be done if --transform was used, or it should be done against the original file size (but it is not saved or kept anywhere, so unlikely to happen soon, I'd have to change the output format)

  2. it should not allow to use --cached with --transform, because caching is not supported in that case (it has no effect)