Open ahjohannessen opened 2 years ago
borg always locks the repo when accessing it.
your problem is due to unsynchronised parallel access.
possible solutions:
--lock-wait 3600
to have borg wait up to an hour while trying to get a lock@ThomasWaldmann As I use rsync.net, I get read-only .zfs snaphots folder with the borg repos in it. Perhaps I could do something with that for the backup restore test?
If it is read-only, borg won't be able to create a lock.
How about trying what I suggested above?
It is a nomad job on a different machine that does the extract and startup of a database, your suggestion assumes the same machine.
.zfs snapshot folder is read-only and immutable. Was thinking of using —bypass-lock with extract for the verification.
If you make sure the repo is read only and in a stable (snapshot) state, there is no risk trying --bypass-lock, but I would not recommend that if that is not the case.
I am sure, .zfs is a special folder with snapshots mounted as readonly. So I can do backup restore tests against the last daily snaphot and point borg to that for the repos and use the —bypass-lock flag with extract
@ThomasWaldmann Seems like doing something along these lines works:
snap="daily_$(date +%Y-%m-%d)"
repo="xyz@xyz.rsync.net:.zfs/snapshot/$snap/borg/abc/repo01"
# transient tmp as it is going to be a throw away docker container running this and repo path also changes each time.
export BORG_BASE_DIR=/tmp
borg --bypass-lock list --last 1 --short $repo | head -n1 | xargs -n1 -I% borg --bypass-lock extract $repo::%
I can do normal borgmatic runs every 20 min and do a full restore test daily using that .zfs snapshot folder trick. That gives me feedback that backups are working.
Have you checked borgbackup docs, FAQ, and open Github issues?
Yes
Is this a BUG / ISSUE report or a QUESTION?
Question
Your borg version (borg -V).
client: borg 1.2.1 server: borg1 1.1.17
Operating system (distribution) and version.
client: Ubuntu 20.04.4 LTS server: rsync.net / FreeBSD
Describe the problem you're observing.
I am having a bit of trouble doing continuous backup restore tests using extract / export-tar due to locks.
The problem I am observing is that I cannot do borg prune / borg compact / borg create / borg check (this is borgmatic default) while doing a borg extract / export-tar, because borg has a lock on the whole repo while performing a backup.
I can understand that prune/compact can conflict with export / export-tar, but a lock is also used during the other commands.
How am I supposed to test my backups if the test causes a backup to fail and a backup causes a backup test to fail? I have systemd timer that runs every 20 minutes.
Is this a case of using
--bypass-lock
while doing extraction?