search

akaihola / hardlinkpy

hardlink.py is a tool to hardlink together identical files in order to save space. Forked from the original version by John L. Villalovos.
http://code.google.com/p/hardlinkpy/
GNU General Public License v2.0
20 stars 8 forks source link

Prevent hardlinking attempt if it will exceed the filesystem maximum #26

Open chadnetzer opened 6 years ago

chadnetzer commented 6 years ago

This was an issue (14) from JohnVillalovos's Google Code repo (brought over to Github), and an issue addressed by @wolfospealain in his PR #3.

The benefit of doing this would be avoiding repeatedly attempting a link and having to unwind when it fails. However, we also want to avoid repeatedly inquiring about the maximum allowed link for a given file.

It should be easy to keep a dictionary for each new st_dev encountered, that queries the maximum links allowed for that device, and using it to avoid linking if the maximum is reached. If a file has reached the maximum allowed links, it may also be beneficial to remove it from consideration for future linking, and allow another equivalent file with less than the maximum to be linked to.

This may seem like overkill, but the maximum nlinks for many common file systems is no more than a 15 or 16 bit number, and there may be reasonable use cases for hardlinkpy that hit that limit.

dsadinoff commented 3 years ago

in addition, there exist filesystems with drastically low max hardlinks, including AWS EFS's 117

Note that a command-line option to override it to be smaller may make sense when the goal is building a "low-count-compatible" tree for export to a different FS.