Suppress "Previously missing file chunk is still missing"?

sshaikh commented 3 weeks ago

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

Question

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

1.4.0

Operating system (distribution) and version.

Debian Bookworm

Hardware / network configuration, and filesystems used.

NA

How much data is handled by borg?

around 250GB

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg check -p -v repo

Describe the problem you're observing.

After a successful repair, the next create is unable to provide chunks for healing (as the item is no longer available). I understand and accept that this means it is lost (and I'm okay with that - these were all transient files) but what are the options to "mark" this as acknowledged? Some ideas:

Delete the individual entry from the backup set for that date
Explicitly mark the file as "known missing"
Restore the chunk from another borg repo (if available - in this case it is)

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

It is reproducible (and I presume by design).

ThomasWaldmann commented 3 weeks ago

One would either need:

to remove all files referencing the lost chunk from all archives,
or remove all archives that have such a file,
or put the missing chunk into the repo and do a borg check --repair. If you can't reproduce the chunk with borg create, guess you could use borg debug commands to fetch it from another repo and put it into this repo and then run repair. Never tried that though.

There is no option to suppress that warning (yet?).

sshaikh commented 3 weeks ago

Can you elaborate on the third option? How would I trace and copy the chunk from the alternative repo?

ThomasWaldmann commented 3 weeks ago

Hmm, just remembered: if the 2 repos were individually initialized, it is unlikely that you have the same chunk in the other repo due to a different chunker secret cutting the chunks differently and also a different id secret computing the chunk ids differently (this is done on purpose, kind of a fingerprinting attack protection).

borg2 will have the concept of "related repos" sharing chunker and id secrets, but not borg 1.x, IIRC.

So, if it is not important, I'ld rather not use the debug commands to fix that.

If that is not the case and you want to do it on your own risk:

https://borgbackup.readthedocs.io/en/stable/usage/debug.html and borg debug --help.

You'll need the chunkid (hex hash, 256bits long) and then get-obj from the other repo and put-obj to this repo.

sshaikh commented 3 weeks ago

Yes, I'm pretty sure they were individually initialised, but I'll check.

I suppose an easy workaround would be to restore the files, perform the backup, and then delete them again?

ThomasWaldmann commented 3 weeks ago

Ah, yes, that should work also. extract from other, create to this, check --repair.

sshaikh commented 2 weeks ago

I went ahead and successfully repaired the repo. Some (perhaps obvious) tips if anyone needs to do the same:

You do not need to restore/extract files to the same place as before (I used a new restore directory within a usual backup location
You don't need to extract all problematic files, just one instance per problematic chunk.
Once you have run the create and repair, you can delete the "temporary" archive if you want.

This exercise actually helped me understand more about how Borg works. Thanks for a great tool!

borgbackup / borg