zstd levels for gitea dump don't seem to work

[joshuataylor]

Description

Using the environment ZSTD_CLEVEL doesn't seem to work, as noted on the initial feature request.

I've tried various levels. Does the go version use these levels?

I have 3 smallish repositories:

Repository Size 943 KiB
Repository Size 857 KiB
Repository Size 6.0 MiB

Command used:

/app/gitea/gitea dump -c /data/gitea/conf/app.ini --file /backups/bk_foo.tar.zst --type tar.zst

Output:

2023/08/22 10:25:36 ...les/setting/cache.go:75:loadCacheFrom() [I] Cache Service Enabled
2023/08/22 10:25:36 ...les/setting/cache.go:90:loadCacheFrom() [I] Last Commit Cache Service Enabled
2023/08/22 10:25:36 ...s/setting/session.go:74:loadSessionFrom() [I] Session Service Enabled
2023/08/22 10:25:36 ...s/storage/storage.go:176:initAttachments() [I] Initialising Attachment storage with type: local
2023/08/22 10:25:36 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/gitea/attachments
2023/08/22 10:25:36 ...s/storage/storage.go:166:initAvatars() [I] Initialising Avatar storage with type: local
2023/08/22 10:25:36 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/gitea/avatars
2023/08/22 10:25:36 ...s/storage/storage.go:192:initRepoAvatars() [I] Initialising Repository Avatar storage with type: local
2023/08/22 10:25:36 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/gitea/repo-avatars
2023/08/22 10:25:36 ...s/storage/storage.go:186:initLFS() [I] Initialising LFS storage with type: local
2023/08/22 10:25:36 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/git/lfs
2023/08/22 10:25:36 ...s/storage/storage.go:198:initRepoArchives() [I] Initialising Repository Archive storage with type: local
2023/08/22 10:25:36 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/gitea/repo-archive
2023/08/22 10:25:36 ...s/storage/storage.go:208:initPackages() [I] Initialising Packages storage with type: local
2023/08/22 10:25:36 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/gitea/packages
2023/08/22 10:25:36 cmd/dump.go:258:runDump() [I] Dumping local repositories... /data/git/repositories
2023/08/22 10:25:36 cmd/dump.go:299:runDump() [I] Dumping database...
2023/08/22 10:25:36 cmd/dump.go:311:runDump() [I] Adding custom configuration file from /data/gitea/conf/app.ini
2023/08/22 10:25:36 cmd/dump.go:327:runDump() [I] Custom dir /data/gitea is inside data dir /data/gitea, skipped
2023/08/22 10:25:36 cmd/dump.go:339:runDump() [I] Packing data directory.../data/gitea
2023/08/22 10:25:36 cmd/dump.go:423:runDump() [I] Finish dumping in file /backups/bk_foo.tar.zst
docker exec -u git -it -w /backups $(docker ps -qf "name=gitea") bash -c   0.01s user 0.01s system 6% cpu 0.297 total

FYI, i also benchmarked the other commands.

Gitea Version

1.20.3

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

2.40.1

Operating System

Docker gitea/gitea

How are you running Gitea?

Docker gitea/gitea

Database

None

[yp05327]

Gitea uses github.com/mholt/archiver/v3 which has been mentioned here https://github.com/go-gitea/gitea/issues/14290#issuecomment-1193314250 And it using github.com/klauspost/compress/zstd to handle tar.zst type. According to the home page of this package, only zstd level 1 and 3 has been implemented.

For now, a high speed (fastest) and medium-fast (default) compressor has been implemented.

• The "Fastest" compression ratio is roughly equivalent to zstd level 1.
• The "Default" compression ratio is roughly equivalent to zstd level 3 (default).
• The "Better" compression ratio is roughly equivalent to zstd level 7.
• The "Best" compression ratio is roughly equivalent to zstd level 11.

So the environment ZSTD_CLEVEL would not have effect, as it is not using zstd command. https://manpages.debian.org/unstable/zstd/zstd.1.en.html