search

ambv / bitrot

Detects bit rotten files on the hard drive to save your precious photo and music collection from slow decay.
MIT License
209 stars 36 forks source link

Ignoring files or directories, or specifying files to be scanned #12

Open pallinger opened 8 years ago

pallinger commented 8 years ago

I know this is quite a big feature, but could be nice to have: I have a lot of small backup files in a directory, and bitrot is progressing really slowly (0.1% in ~20 hours). One simple implementation would be for bitrot to accept a predefined file-list, which could be generated by find or similar, that has already exclude options. I may start to work on it, if I have the time. :)

ambv commented 8 years ago

Wow, 0.1% in 20 hours is pretty awful. As for a predefined file list, I'm thinking about an ignore list instead. If you really want to track known files, you can easily do it with md5sum and friends.

pallinger commented 8 years ago

I do not mean tracking known files, I thought about being able to use bitrot as part of a toolchain. E.g. find -type f -size +1M | grep -v '~$' | bitrot --file-list-from-stdin. So I did not think about using a static file list, but an on-the-fly-generated one. Of course, a --exclude-file-list-from-stdin might also do the same trick.

benshep commented 7 years ago

I made a fork of the project to do this: https://github.com/benshep/bitrot

You can use --file-list (accepts a file or stdin) and --exclude-list (accepts files only, including wildcards).

huyz commented 4 years ago

@benshep Are you planning to update your fork for bitrot 1.0.0? (I need to exclude some Maildir directories that contain files that get frequently modified by IMAP syncing.)

benshep commented 4 years ago

I think it should be fairly straightforward to make a pull request from my fork. I'll try to find some time for that this week.

p1r473 commented 4 years ago

@huyz My fork has been updated to 1.0 now Took me all day! I'd love some help unit testing, if anyone is able to help me test.

@pallinger My fork has both inclusion and exclusion lists

benshep commented 4 years ago

Great! I guess we can use your fork @p1r473 - thanks. That saves me a bunch of effort, and I'm certain you'll have done it better than I would ☺

ambv commented 4 years ago

@p1r473, wow, you updated the fork to 1.0? That's amazing. If you'd like to submit your support for ignoring files as a pull request, I'll accept it.

huyz commented 4 years ago

@p1r473 Great! I can't find the usage instructions for how to include and exclude. How does one do it?

p1r473 commented 4 years ago

@p1r473, wow, you updated the fork to 1.0? That's amazing. If you'd like to submit your support for ignoring files as a pull request, I'll accept it.

There are just so many changes from my fork... I think I've more than doubled the line count

p1r473 commented 4 years ago

@p1r473 Great! I can't find the usage instructions for how to include and exclude. How does one do it?

py bitrot.py --exclude-list exclude.txt --include-list include.txt Include takes precedence over exclude

huyz commented 4 years ago

There are just so many changes from my fork... I think I've more than doubled the line count

@p1r473 You have made a lot of changes. Can you describe all the changes you've made? (Maybe add them to your version of the README?)

p1r473 commented 4 years ago

There are just so many changes from my fork... I think I've more than doubled the line count

@p1r473 You have made a lot of changes. Can you describe all the changes you've made? (Maybe add them to your version of the README?)

They are already in my readme

huyz commented 4 years ago

@p1r473 You have made a lot of changes. Can you describe all the changes you've made? (Maybe add them to your version of the README?)

They are already in my readme

I meant, as a separate section so that it’s clear what features your fork adds to the original repo