search

whatwg / compression

Compression Standard
https://compression.spec.whatwg.org/
Other
85 stars 21 forks source link

zstd support #54

Open ricea opened 1 year ago

ricea commented 1 year ago

Issue for discussion of adding support for the zstd format to the API.

ricea commented 1 year ago

Currently libzstd is not linked into Chromium. It would add roughly 700KB to the binary size (maybe can be trimmed a bit). This would be a hard sell if the compression standard is the only user.

evanstade commented 1 year ago

It seems that you did manage to trim the size a bit as this change only added 38k. Was that just by dropping everything besides decompression, or were there other tricks you made use of?

On the storage team we are also interested in using zstd for compressing IndexedDB values before sending them over IPC (or storing as blobs). Adding the compression half of zstd appears to add another 200kB.

So that's at least two users of zstd.

ricea commented 1 year ago

We didn't do anything special in Chromium to reduce the size increase, except for only compiling in the decompression code. I think we were just lucky.

I would like to move this specification to WHATWG before we add new features, since it is now implemented in Firefox and Safari and we would like to have consensus.

That means it will probably be some months before there's movement on adding zstd.

saschanaz commented 1 year ago

I would like to move this specification to WHATWG before we add new features, since it is now implemented in Firefox and Safari and we would like to have consensus.

Why would it take that much to move to WHATWG?

telgareith commented 10 months ago

Note: Chrome/Chromium, Firefox, and Opera have agreed to add zstd to content-encoding options. Chrome is planning on General Availability in 118.

Jedipedia commented 4 months ago

Another note: Firefox now supports Content-Encoding: zstd as of version 126 but there's still no support for new DecompressionStream("zstd") in either Firefox or Chromium.

xpl commented 3 months ago

Please add it to the specification, as it seemingly blocks from implementing new DecompressionStream("zstd") in browsers:

https://issues.chromium.org/issues/348499728

Screenshot 2024-06-24 at 12 10 32
saschanaz commented 3 weeks ago

https://github.com/whatwg/compression/issues/34#issuecomment-2340077884

Just as brotli, we'd be open to ship only DecompressionStream support for zstd, as @jesup reported 600KB size increase to ship the compression side of the library.

ricea commented 3 weeks ago

#34 (comment)

Just as brotli, we'd be open to ship only DecompressionStream support for zstd, as @jesup reported 600KB size increase to ship the compression side of the library.

Thanks. I think this will also be Blink's position.

nektro commented 1 week ago

Bun would be interested in supporting this since we already ship with zstd support internally for other APIs