pkolaczk / fclones

Efficient Duplicate File Finder
MIT License
1.96k stars 75 forks source link

isolate behavior removes dupes same volume #97

Closed mlybarger closed 2 years ago

mlybarger commented 2 years ago

when using --isolate for two directories, duplicate files within the directories are also being deleted. my understanding is the files w/i the volumes aren't considered for duplicate removal.

`
cd /mnt/media mkdir test-a mkdir test-b echo "test-one" > test-a/test-1 echo "test-one" > test-a/test-2 echo "test-one" > test-b/test-1 echo "test-one" > test-b/test-2 echo "test-two" > test-a/test-3 echo "test-two" > test-a/test-4 echo "test-two" > test-b/test-4 fclones group --isolate /mnt/media/test-a /mnt/media/test-b > test.txt fclones remove --dry-run <test.txt [2022-02-15 12:53:15.823] fclones: info: Started deduplicating (dry run) rm /mnt/media/test-a/test-4 rm /mnt/media/test-a/test-2 rm /mnt/media/test-b/test-4 rm /mnt/media/test-b/test-1 rm /mnt/media/test-b/test-2 [2022-02-15 12:53:15.826] fclones: info: Would process 5 files and reclaim 45 B space

`

pkolaczk commented 2 years ago

The --isolate option makes duplicate files within the same root counted as one. But in this case you have same files under different roots, so they are properly treated as duplicates.

pkolaczk commented 2 years ago

After thinking about it a bit more, indeed I think the removal behaviour is wrong. It should only remove files from one root in this case and leave the other root untouched.

pkolaczk commented 2 years ago

Still not perfect, because reporting of the number of files to be removed in the report is wrong.

pkolaczk commented 2 years ago

@mlybarger But remove/link/move/dedupe commands should work correctly now.

pkolaczk@p5520:~/Projekty/fclones (dedupe_isolated_roots)$ cargo run -- group --isolate test/root1 test/root2 -o report.txt
   Compiling fclones v0.17.1 (/home/pkolaczk/Projekty/fclones)
    Finished dev [unoptimized + debuginfo] target(s) in 2.61s
     Running `/home/pkolaczk/.cargo/target/debug/fclones group --isolate test/root1 test/root2 -o report.txt`
[2022-02-26 17:02:34.493] fclones:  info: Started grouping
[2022-02-26 17:02:34.657] fclones:  info: Scanned 6 file entries
[2022-02-26 17:02:34.658] fclones:  info: Found 4 (16 B) files matching selection criteria
[2022-02-26 17:02:34.661] fclones:  info: Found 3 (12 B) candidates after grouping by size
[2022-02-26 17:02:34.665] fclones:  info: Found 3 (12 B) candidates after grouping by paths and file identifiers
[2022-02-26 17:02:34.692] fclones:  info: Found 3 (12 B) candidates after grouping by prefix
[2022-02-26 17:02:34.694] fclones:  info: Found 3 (12 B) candidates after grouping by suffix
[2022-02-26 17:02:34.696] fclones:  info: Found 3 (12 B) redundant files
pkolaczk@p5520:~/Projekty/fclones (dedupe_isolated_roots)$ cat report.txt 
# Report by fclones 0.17.1
# Timestamp: 2022-02-26 17:02:34.697 +0100
# Command: /home/pkolaczk/.cargo/target/debug/fclones group --isolate test/root1 test/root2 -o report.txt
# Base dir: /home/pkolaczk/Projekty/fclones
# Found 1 file groups
# 12 B (12 B) in 3 redundant files can be removed
6109f093b3fd5eb1060989c990d1226f, 4 B (4 B) * 4:
    /home/pkolaczk/Projekty/fclones/test/root1/foo1
    /home/pkolaczk/Projekty/fclones/test/root1/foo2
    /home/pkolaczk/Projekty/fclones/test/root2/foo1
    /home/pkolaczk/Projekty/fclones/test/root2/foo2
pkolaczk@p5520:~/Projekty/fclones (dedupe_isolated_roots)$ cargo run -- remove --dry-run <report.txt
    Finished dev [unoptimized + debuginfo] target(s) in 0.04s
     Running `/home/pkolaczk/.cargo/target/debug/fclones remove --dry-run`
[2022-02-26 17:03:09.129] fclones:  info: Started deduplicating (dry run)
rm /home/pkolaczk/Projekty/fclones/test/root2/foo1
rm /home/pkolaczk/Projekty/fclones/test/root2/foo2
[2022-02-26 17:03:09.258] fclones:  info: Would process 2 files and reclaim 8 B space
pkolaczk commented 2 years ago

Closing this and created a separate issue #99