search

whatwg / compression

Compression Standard
https://compression.spec.whatwg.org/
Other
85 stars 21 forks source link

Support for decompressing multi-member gzip files? #42

Open ikreymer opened 2 years ago

ikreymer commented 2 years ago

The GZIP spec includes support for one or more members (A gzip file consists of a series of "members" (compressed data sets). but this spec currently states A gzip stream may only contain one "member"..

This is a request to be able to support decompressing multi-member gzip files to support use cases that depend on multi-member gzip (such as parsing ISO WARC).

Alternatively, if a way to get the unused data as in #39 would allow the developer to implement this manually, those ideal solution for my use case be native support for a multi-member DecompressionStream.

Could provide a proposed spec if there would be interest..

ricea commented 2 years ago

Interesting! I didn't know there was demand for this. I can't commit to implementing it, but I would like to see your proposed spec.

evertheylen commented 1 day ago

I recently had the same problem and made this. It works by inspecting the chunks for potential new streams (using the magic 1f8b08 bytes), splitting when necessary, and re-creating a DecompressionStream when it crashes. Maybe it helps some of you!