kjdev / php-ext-zstd

Zstd Extension for PHP
MIT License
201 stars 27 forks source link

Unspecified buffer size for decompression. #32

Open TheBlackPlague opened 4 years ago

TheBlackPlague commented 4 years ago

This extension doesn't save a decompressed size field value making it impossible to decompress strings compressed by this using any other binding such as ZstdNet.

remicollet commented 4 years ago

This extension support the various Zstd compression modes

Looks like the issue is that Zstdnet doesn't support streaming mode.

P.S. notice: this extension simply provides bindings on the libzstd, and implement most of its features

remicollet commented 4 years ago

From http://facebook.github.io/zstd/zstd_manual.html (for ZSTD_getFrameContentSize)

note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode. When return==ZSTD_CONTENTSIZE_UNKNOWN, data to decompress could be any size. In which case, it's necessary to use streaming mode to decompress data.

TheBlackPlague commented 4 years ago

From http://facebook.github.io/zstd/zstd_manual.html (for ZSTD_getFrameContentSize)

note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode. When return==ZSTD_CONTENTSIZE_UNKNOWN, data to decompress could be any size. In which case, it's necessary to use streaming mode to decompress data.

This extension support the various Zstd compression modes

  • simple / buffered, which save decompressed size needed by simple decompression mode
  • streaming, which doesn't, and thus requires stream decompression mode

Looks like the issue is that Zstdnet doesn't support streaming mode.

P.S. notice: this extension simply provides bindings on the libzstd, and implement most of its features

Thank you for the information. However, we were using non-stream based compression and still lacking the decompressed size. It would be nice to add it in for non-stream based compression.