search

borgbackup / borg

Deduplicating archiver with compression and authenticated encryption.
https://www.borgbackup.org/
Other
11.2k stars 742 forks source link

borg bisect #5612

Open anarcat opened 3 years ago

anarcat commented 3 years ago

I often find myself in a situation where I discover I mistakenly destroyed data, but several days after the fact. For example, today I remember doing an operation that destroyed content, "some time ago" (days? weeks?) which I am pretty sure could be restored by borg. I just don't remember when I did so. Ideally, I would have restored the content right after I did that (using the latest snapshot), but I wasn't in a rush, and wasn't sure I actually lost data then. Now I'm sure, but I don't know when!

Now I know I can list my most recent snapshots with:

borg list REPO

Then I can look inside those archives with:

borg list REPO::ARCHIVE PATH

Now, I know that I do borg mount to setup a FUSE filesystem representing the entire archive with all the snapshots, but that's awfully slow. Wouldn't it be great if there was a way to say "find me the latest snapshot with that file still intact"? We could even do crazy stuff like "find that file with such size" if it was zero'd or overwritten in some way, for example, or search for timestamps...

A workaround I've found is:

borg list REPO --format "{name}{NUL}" | tac | while read archive ; do borg list REPO::$archive PATH | grep . && echo $archive has the file ; done

Hardly ideal, and definitely not bisect (although I guess I could use shuf to get a better luck - is there such a thing as bisect(1)? how would that even make sense?)....

(also, while i'm here: it's too bad borg-list doesn't exit with an error if it can't find the provided pattern... that would avoid the silly | grep . command above...)

ThomasWaldmann commented 3 years ago

How about using the versions view to easily see all file versions (and use timestamps to determine the archive you're searching for)?:

borg mount -o versions ...

If it's only a single file you search, you can also just copy it from that mount.

About speed: borg archives are linear, so if you need to open and search through multiple (or even many) of them, it will always be slow.

anarcat commented 3 years ago

How about using the versions view to easily see all file versions (and use timestamps to determine the archive you're searching for)?

Also slower...

About speed: borg archives are linear, so if you need to open and search through multiple (or even many) of them, it will always be slow.

I guess what I'm looking for is the equivalent of the above shell command. I understand searching linearly is slow, but i want to be as fast as possible to that slowness, if you see what i mean. From what i understand, fuse mounts do way more work than strictly necessary, even given the linear data structures...

ThomasWaldmann commented 3 years ago

BTW, you can limit borg mount to work on a subset of archives (--first, --last, --prefix, --glob-archives) and also to work on a subset of the archived files (giving PATH or using include/exclude patterns).

It should get quite faster by that and will only build a fs with the interesting stuff (and quickly skipping the other stuff).

anarcat commented 3 years ago

ah, that's certainly useful. maybe we should add an examples section to the borg-mount manpage...