markfasheh / duperemove

Tools for deduping file systems
GNU General Public License v2.0
816 stars 81 forks source link

Provide option to cross filesystem boundaries #178

Closed etskinner closed 4 years ago

etskinner commented 7 years ago

Man page shows:

-x Don't cross filesystem boundaries, this is the default behavior since duperemove v0.11. The option is kept for backwards compatibility.

In btrfs, subvolumes act like different mount points (boundaries), but they could still benefit from de-duping. For example, consider a directory in a btrfs filesystem with subvolumes 1 and 2, where there exist duplicate files foo in each:

.
├── 1
│   └── foo
│   └── bar
└── 2
    ├── foo
    └── baz

It would be useful to have an option that would allow for filesystem boundary crossing (something like --nx or --subvolume?). Ideally, duperemove would check if subvolumes that it recurses into are part of the same overall filesystem, and provide warnings if not.

leggewie commented 6 years ago

In the meantime I suppose what you want can be achieved by bind-mounting your subvolumes somewhere appropriate (/mnt or /tmp) and then run duperemove without -x on that mount.

lorddoskias commented 4 years ago

Current dedupe support in the kernel doesn't support dedupe across mountpoints so it makes no sense to support this in duperemove at present time.