ThomasWaldmann
commented
9 months ago Hmm, IIRC borg always sequentially goes over the full archive, matching the pattern to each encountered item path and extracting it if it matches.
So it is very likely (considering the usual borg recursion behaviour at borg create time) that initially the path pattern will not match (close to 0 transfer rate), then it will match for a while (high transfer rate, actually extracting stuff), then it will stop matching for the rest of the archive (close to 0 transfer rate).
The "close to zero" rate is when borg only reads metadata from the archive metadata stream, not file content data.
So, guess we could "short circuit" that and stop borg extract when it leaves the desired folder. That would only work for the most simple use case (as you had) and only for the current single-threaded borg internal recursor though.
Multi-threading (in future) and also an external recursor that behaves differently would be problematic.
So, could it be borg is already finished extracting the data you wanted?
An alternative explanation for widely varying transfer rate can be file size:
- if borg extracts big files, you'll see a high transfer rate, because there is minimal overhead and content size determines processing time.
- if borg extracts very small files, you'll see a much lower transfer rate, because overhead determines per-file processing time.
Yet another explanation of course can be outside factors:
- your network connection being used otherwise also
- your disks being actively used otherwise also
Frostregen
Have you checked borgbackup docs, FAQ, and open GitHub issues?
Yes
Is this a BUG / ISSUE report or a QUESTION?
ISSUE
System information. For client/server mode post info for both machines.
Your borg version (borg -V).
Both client and server machine run: borg 1.2.0
Operating system (distribution) and version.
Client: Ubuntu 22.04.2 Server: Ubuntu 22.04.3
Hardware / network configuration, and filesystems used.
Client: 8C/16T CPU, 64GB RAM, 12x 20TB HDD in RAID6 Server: 6C/12T CPU, 128GB RAM, 10x 10TB HDD in RAID6
How much data is handled by borg?
Full borg commandline that lead to the problem (leave away excludes and passwords)
borg extract --progress ssh://<USER>@<IP>:<PORT>/path/to/backup/::2024-01-24-1706130008 /path/into/subfolderDescribe the problem you're observing.
Initially (on the first day) when extracting, the speed is as expected around 100MB/s (on a 1 Gbit/s line).
After some time the speed crawls down to 20KB/s, or even 0. Rarely it also goes back up to around 100MB/s.
This makes it look like it will take assumed ~50 days, instead of the anticipated ~2 days. (The subfolder should be around 10 TB)
I'm suspecting this has to do with specifying the "/path/into/subfolder". Compared to running a plain borg extract of the full archive (no subfolder specified), I could not observe that drop in speed.
The "fun" part is, that the repository also has some Integrity errors, so after ~21% of running a full extract (
borg extract --progress ssh://<repopath>::2024-01-24-1706130008), I get:This is why I tried to extract subfolders at all.
I ran
borg check --verify-databefore too, which gave me:Followed by a
borg check --repair --progress ssh://<repopath>::2024-01-24-1706130008- But I still got the same IntegrityError when trying to extract everything afterwards.Would a
borg check --repair --verify-data ...actually fix that issue?I am a bit lost currently, and would be happy for any pointers on how to proceed.
Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes. Always happens in a similar fashion. (Cancel and retry). Also tried borg mount + rsync, but that was similarly slow.
Include any warning/errors/backtraces from the system logs
Already above