whatwg / compression

Compression Standard
https://compression.spec.whatwg.org/
Other
82 stars 21 forks source link

Support "deflate-raw" format #25

Closed ricea closed 2 years ago

ricea commented 4 years ago

For applications such as reading/writing zip files it's necessary to have access to the raw deflate stream without any headers or footers. We should add a "deflate-raw" format to support this.

See previous discussion at https://github.com/ricea/compressstream-explainer/issues/8.

taralx commented 3 years ago

Note that zip files can be decompressed (and checksummed!) using gzip:

const GZIP_HEADER = Uint8Array.from([
  31, 139, // gzip magic
  8, // deflate
  0, // no extra fields
  0, 0, 0, 0, // mtime (n/a)
  0, 0, // extra flags, OS
]);

function decompressZip(stream, crc, uncompressedSize) {
  return stream.pipeThrough(new TransformStream({
    start(controller) {
      controller.enqueue(GZIP_HEADER);
    },
    flush(controller) {
      const tmp = new DataView(new ArrayBuffer(8));
      tmp.setUint32(0, crc, true);
      tmp.setUint32(4, uncompressedSize, true);
      controller.enqueue(new Uint8Array(tmp.buffer));
    }
  })).pipeThrough(new DecompressionStream('gzip'));
}
taralx commented 3 years ago

Note that for the creation side, you need CRC, so you may as well use gzip there too.

ricea commented 3 years ago

@taralx That's impressive, but I'd prefer it if people didn't have to put together a fake gzip wrapper just to decode a zip file.

I like the CRC trick. It doesn't look like there'd be an easy way to get the same functionality with "deflate-raw".

taralx commented 3 years ago

In decompression it can be a constructor option. In compression though, I'm not sure either.

panva commented 2 years ago

I am likewise looking for support of 'inflate-raw' and 'deflate-raw' formats.

nikhil1511 commented 2 years ago

We are also interested in having support for "deflate-raw" as it is required for zipping. With "gzip" or "deflate" formats, additional workarounds would be required which is not very neat. So, it would be super helpful if raw compression format can be supported. Thanks!

yutakahirano commented 2 years ago

Chrome is interested in this feature, and wants to add this feature. Are there any opinions from other browser vendors?

yutakahirano commented 2 years ago

Working at #43. Any feedbacks will be appreciated.

yutakahirano commented 2 years ago

@ricea I cannot close this, but I think this can be closed now.

ricea commented 2 years ago

Done.