bp74 / Zstandard.Net

A Zstandard wrapper for .Net
Other
135 stars 26 forks source link

Unable to decompress using a synchronization context #20

Open chema3520 opened 4 years ago

chema3520 commented 4 years ago

Hello, I'm working in a client/server project where I'm providing zstd support on the client side. First time I receive a compressed payload I'm able to decompress succesfully, however the second time the decompression fails.

I'm initializing the following objects in the class constructor and reusing them each time a decompression is required: _cache = new MemoryStream(); _decompressionStream = new ZstandardStream(_cache, CompressionMode.Decompress, true);

Code for decompression is as follows:

{ _cache.Write(input, 0, input.Length); _cache.Position = 0; var target = new MemoryStream(); _decompressionStream.CopyTo(target, uncompressedLength); _decompressionStream.Flush(); var result = target.ToArray(); _cache.SetLength(0); return result; }

The second time code is executed the result is a byte[0] array. It stays the same even if I close or dispose of the ZStandardStream object. It's important to note that the data received in both executions is the following:

First execution: new byte[] { 40, 181, 47, 253, 0, 72, 12, 4, 0, 212, 6, 86, 0, 0, 0, 12, 8, 7, 64, 255, 1, 80, 128, 2, 88, 16, 18, 13, 86, 97, 114, 105, 97, 98, 108, 101, 95, 110, 97, 109, 101, 26, 34, 17, 115, 101, 115, 115, 105, 111, 110, 95, 118, 115, 42, 50, 0, 58, 3, 100, 101, 102, 68, 32, 18, 5, 86, 97, 108, 117, 101, 26, 20, 0, 0, 0, 13, 10, 8, 118, 101, 114, 115, 105, 111, 110, 0, 10, 7, 56, 46, 48, 46, 49, 57, 0, 1, 0, 0, 0, 14, 15, 0, 0, 0, 11, 8, 3, 16, 2, 26, 8, 8, 4, 18, 4, 8, 2, 24, 0, 6, 0, 46, 193, 0, 21, 62, 93, 48, 184, 77, 143, 117, 102, 126, 145, 167, 19 }

Second execution: new byte[] { 28, 6, 0, 100, 9, 60, 45, 80, 192, 1, 88, 80, 18, 7, 67, 104, 97, 114, 115, 101, 116, 26, 34, 14, 67, 72, 65, 82, 65, 67, 84, 69, 82, 95, 83, 69, 84, 83, 42, 2, 99, 115, 50, 5, 109, 121, 115, 113, 108, 45, 80, 128, 48, 88, 16, 18, 11, 68, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 26, 81, 192, 1, 88, 80, 18, 17, 68, 101, 102, 97, 117, 108, 116, 32, 99, 111, 108, 108, 97, 116, 105, 111, 110, 26, 3, 99, 111, 108, 55, 2, 80, 10, 88, 16, 18, 6, 77, 97, 120, 108, 101, 110, 26, 51, 117, 116, 102, 56, 109, 98, 52, 0, 10, 14, 85, 84, 70, 45, 56, 32, 85, 110, 105, 99, 111, 100, 101, 0, 10, 19, 95, 48, 57, 48, 48, 95, 97, 105, 95, 99, 105, 0, 10, 1, 4, 16, 0, 106, 11, 84, 123, 28, 94, 34, 76, 182, 175, 131, 175, 38, 176, 226, 20, 132, 11, 50, 142, 174, 48, 3, 108, 250, 1, 133, 44, 1, 7, 96, 56, 246, 238, 157, 129, 69, 192, 158, 5, 33 }

I'm aware that no header info is present in the second set of data since it is expecting that a synchronization context is already in place and data is reused. May I know if this is supported by the library and if so how can I make it work?

Thanks.

Brain2000 commented 3 years ago

The ZStd library is a streaming library that keeps track of state between calls to it, so when you create a ZStandardStream( ) object, it is meant to receive a continuous stream of compressed data from a source, not individual blocks with clear start and stop points. If you want to do that, I believe you might have to re-create the ZStandardStream object each time.

Having the synchronization context just makes it so you aren't using the same object with multiple threads, but if you re-create the ZStandardStream( ) object in each separate thread, then you don't have to worry about multi-threading, because the objects will not be used across threads.