denoland / std

The Deno Standard Library
https://jsr.io/@std
MIT License
2.99k stars 596 forks source link

Tar doesn't support sparse tar files #3127

Open oscarotero opened 1 year ago

oscarotero commented 1 year ago

Describe the bug

I'm using this code to decompress tar.gz files: https://github.com/oscarotero/dbin/blob/d12abeb82b72ba079cca33eef4621dfd55b57047/mod.ts#L161-L186

This code is used to download binary files from the assets of github releases so they can be executed. One example is Pagefind that is downloaded according to the this configuration in Lume.

According to this comment: https://github.com/lumeland/lume/issues/362#issuecomment-1396404347 the function to uncompress tag files doesn't support sparse files, making the binary file not executable.

Environment

This specific issue only happens on MacOS on M1/ARM because the binary targeted for this architecture is compressed as sparse tar. I cannot reproduce this issue because my computer is MacOS on Intel.

bglw commented 1 year ago

Some extra context: Here is a repo that reproduces the error that was hit here: https://github.com/bglw/deno-tar-repro

The issue can be reproduced on any system by running the file above, which will output:

$ deno run file.ts

✅ Granted read access to "pagefind-v0.10.7-aarch64-apple-darwin.tar".
✅ Granted read access to "pagefind-v0.10.6-aarch64-apple-darwin.tar".

Standard tar file contents:
• pagefind

Sparse tar file contents:
• PaxHeader/pagefind
• GNUSparseFile.0/pagefind

(Both should extract as one file `pagefind`)

This specific issue only happens on MacOS on M1/ARM

Actually, this issue actually could occur anywhere — it's just sporadic since a tar file isn't guaranteed to generate this way. It could happen using the system tar on any macOS installation, or it could happen on any other platform if calling tar with the --sparse flag. In this case the ARM build just happened to roll the sparse dice (though it was still being generated from an intel mac image.) The ticket should be tagged as OS: All

bglw commented 1 year ago

NB: Since Deno std is a loose port of the Golang std, it should be noted that this bug was indeed fixed in Go's archive package: https://github.com/golang/go/issues/3864 — commit: https://github.com/golang/go/commit/730db0affc642530daf9129f4fbc89a4e40f9c95