Support zstd compressed tarballs in .chezmoiexternal.toml

d0b3rm4n commented 1 year ago

Is your feature request related to a problem? Please describe.

https://github.com/ankitects/anki/releases releases tarballs zstd compressed (.zst), I would like to install their software with an entry in .chezmoiexternal.toml but it seems there is no support for that compression format.

The following error is given:

chezmoi: .Software/anki/2.1.54-qt6: https://github.com/ankitects/anki/releases/download/2.1.54/anki-2.1.54-linux-qt6.tar.zst: unknown archive format

with this configuration:

[".Software/anki/2.1.54-qt6"]
type="archive"
url="https://github.com/ankitects/anki/releases/download/2.1.54/anki-2.1.54-linux-qt6.tar.zst"

Describe the solution you'd like

Best solution would be that one can add just:

[".Software/anki/2.1.54-qt6"]
type="archive"
url="https://github.com/ankitects/anki/releases/download/2.1.54/anki-2.1.54-linux-qt6.tar.zst"

to .chezmoiexternal.toml and based on the extention .zst it does the right thing. Adding also the format=zstd hint would be acceptable, if auto detection is not possible.

Since it is an rather new format (not sure since when it is also supported in tar )would it be possible maybe to add an option archive.extract_option (or something like that) so that one could configure:

[".Software/anki/2.1.54-qt6"]
type="archive"
url="https://github.com/ankitects/anki/releases/download/2.1.54/anki-2.1.54-linux-qt6.tar.zst"
archive.extract_option=--zstd

So the tar command would use that extract option instead of the usual suspects like -z or -J. This would make it quite flexible and future proof, in case tar would add support also for other algorithms in future. And the feature would depend on when ever the local tar version supports it, no need to test from chezmoi.

Describe alternatives you've considered

Of course I can download it by hand and extract it by hand, but chezmoi is supposed to support me with that step.

halostatue commented 1 year ago

There are two possible enhancements here. Right now, externals are managed entirely by internal operations and not external calls. That is, there is no call to tar xf at any point, but chezmoi handles the decompression itself.

Add zstd compression support to chezmoi. This should be pretty easy to do with https://github.com/klauspost/compress (https://pkg.go.dev/github.com/klauspost/compress/zstd) and modifying the current flow. The hardest part would be making sure that there’s good tests for it.
Add explicit command support to external handling. This would be harder, but potentially more flexible.

twpayne commented 1 year ago

In the short term, you should be able to use a filter to decompress the archive:

[".Software/anki/2.1.54-qt6"]
    type = "archive"
    url = "https://github.com/ankitects/anki/releases/download/2.1.54/anki-2.1.54-linux-qt6.tar.zst"
    format = "tar"
    filter.command = "zstd"
    filter.args = ["-d"]

d0b3rm4n commented 1 year ago

In the short term, you should be able to use a filter to decompress the archive:

[".Software/anki/2.1.54-qt6"]
    type = "archive"
    url = "https://github.com/ankitects/anki/releases/download/2.1.54/anki-2.1.54-linux-qt6.tar.zst"
    format = "tar"
    filter.command = "zstd"
    filter.args = ["-d"]

Oh this is how that works with filters, to be honest I didn't fully understand it from the documentation. My understanding was it would somehow filter the file names to be extracted. But could not image how that should work. It didn't come to my mind it could be used for decompressing the tarball first 🤦‍♂️. This fully covers my request to be future proof.

Tried it out, yes works like a charm, but also thanks a lot for the quick fix :+1: looking forward for the next release.

twpayne / chezmoi