search

pkolaczk / fclones

Efficient Duplicate File Finder
MIT License
1.91k stars 71 forks source link

Warning message confusing when physical file data location not available #83

Closed paony closed 2 years ago

paony commented 2 years ago

I get the following messages running fclones group yyy :

[2021-10-24 14:33:02.212] fclones: warn: Failed to fetch extents for file : Operation not supported (os error 95)

Maybe it's harmless, my problem is that I don't know what it means. I am using version: fclones 0.17.0 system is xubuntu 20.04 (everything upgraded) I run fclones on aboaut 2TiB of data - 200000 files of all sizes filesystem is zfs (no mirror or raid but encrypted) disk is spinning disk 4TiB

Warning occurs on 10 files of the 200000 Any ideas? Could the warning message be somewhat more verbose? Can I just ignore it?

By the way - I have run a brief speed comparison on that data above - here are my results: (Intel quad core - 16GB ram - disk cache fully loaded from previous operations) fclones group xxx 34 min rdfind xxx 81 min jdupes -S -M -Q -r xxx 90 min rmlint -T df xxx 134 min Pretty impresive!!!

jvirkki commented 2 years ago

Since you're doing comparisons, if you have a chance to try out https://github.com/jvirkki/dupd on your particular data set, would be interesting to see (dupd scan -p xxx).

pkolaczk commented 2 years ago

This is generally harmless. It means your file system doesn't support getting physical locations of files through the sysctl fiemap API, hence performance is probably much slower than it could be. Anyway glad it is fast despite that optimisation not kicking in.

I'm leaving this open to improve logging.

dsully commented 2 years ago

This is happening for me with ZFS on Ubuntu 20.10 FWIW. Let me know if I can get any more information for you.

pkolaczk commented 2 years ago

I have everything I need. I need to change the message.

paony commented 2 years ago

Issue is clear for fclones then.

Might be an openZFS bug ...

Thanks for the quick reaction