oleg-st / ZstdSharp

Port of zstd compression library to c#
MIT License
200 stars 29 forks source link

Return available data from stream, before waiting for more #33

Closed LinusKardellInfobric closed 5 months ago

LinusKardellInfobric commented 5 months ago

Previously, Read and ReadAsync would only return once the output buffer had been filled, or the inner stream returned 0 (i.e. when the inner stream ended). That meant it would not return already decoded data until it got more data from the inner stream, even if no more data was currently available. Now, instead:

This does have the drawback that each call to Read will only call innerStream.Read at most once, so the amount of data the caller can get from a single Read call is limited by inputBufferSize.

Fixes #32

oleg-st commented 5 months ago

inputBufferSize = bufferSize > 0 ? bufferSize : (int) Methods.ZSTD_CStreamInSize().EnsureZstdSuccess();

Looks like it should be ZSTD_DStreamInSize instead of ZSTD_CStreamInSize

oleg-st commented 5 months ago

Some tests have failed, such as RoundTrip_BatchToStreaming

LinusKardellInfobric commented 5 months ago

Looks like it should be ZSTD_DStreamInSize instead of ZSTD_CStreamInSize

Unrelated to this change, but OK, I changed that.

Some tests have failed, such as RoundTrip_BatchToStreaming

Turns out when we're completely done with a frame, calling DecompressStream again with no input will return 9 (the minimum possible ZSTD frame size) rather than 0. I've changed it to only keep the result of DecompressStream calls that made some progress.

oleg-st commented 5 months ago

@LinusKardellInfobric Thanks!