borgbackup / borg

Deduplicating archiver with compression and authenticated encryption.
https://www.borgbackup.org/
Other
11.06k stars 739 forks source link

Borg "fat" binaries extract "many" files before running: should TMPDIR be a tmpfs? #8008

Closed trendymail closed 4 hours ago

trendymail commented 9 months ago

Hello!

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yup! Docs, FAQ and Issues.

Every time I read an issue, I learn something... ^^

Is this a BUG / ISSUE report or a QUESTION?

Just a question

Your borg version (borg -V).

borg 1.2.7 ("fat") binaries => borg-linuxold64 and borg-linux64.

Describe the problem you're observing.

Not really a problem, I think that "fat" binaries need to extract some files to be "compatible".

Long story short

Each time a "fat" binary is launched, it extracts some data in order to run (so far so good).

But, can these needed files be stored on an tmpfs?

I do actually set TMPDIR to a tmpfs dir.

Results are (very) good: launch time is divided by 2.

But, since TMPDIR is needed for other files, is 1 GB is enough?

----

Please simply send me a "RTFM" if I missed something. ^^

Have a great night! :)

infectormp commented 9 months ago

I suppose no one has counted how much space needed for this, but

https://borgbackup.readthedocs.io/en/stable/installation.html#standalone-binary

Note that the binary uses /tmp to unpack Borg with all dependencies. It will fail if /tmp has not enough free space or is mounted with the noexec option. You can change the temporary directory by setting the TEMP environment variable before running Borg.

ThomasWaldmann commented 9 months ago

There are 2 borg "fat binary" builds:

If you extract the tgz, you will see the amount and total size of the extracted files for both cases.

If you care a lot for speed, use the tgz and just extract it once.

trendymail commented 8 months ago

@infectormp: many thanks for the RTFM link (I overlooked part "[...] unpack Borg with all dependencies."). @ThomasWaldmann: I was not aware that tgz files were "unpacked" versions of standalone binaries. ;)

Using tgz versions did not yield much performance differences but it allows to skip extraction each time Borg is called.

But hey, "Every Small Step Counts"... ^^

Thank you for your time and your answers, I really appreciate!

ThomasWaldmann commented 2 months ago

TODO: add a note to the borg binaries 00_README.txt that the single file version will always extract to temp and that the tgz single directory version is already extracted and does not need to do that.