search

szcompressor / SZ

Error-bounded Lossy Data Compressor (for floating-point/integer datasets)
http://szcompressor.org
Other
155 stars 56 forks source link

Error bound rate #118

Closed mearcla closed 2 months ago

mearcla commented 2 months ago

Hello, please, I need to understand the relation between the value of error bound and the compression rate. what is the difference between using 0.5,0.005 or set the error bound = Zero ? is there any paper that explain that?

disheng222 commented 2 months ago

SZ is a lossy compressor. Please don't set error bound to zero. We didn't test this case, which may cause unexpected result I guess. Using different error bounds means the reconstructed data after running the compression and decompression will have different levels of data distortion. 0.005 means the difference between original data vs. reconstructed data will be always within the 0.005. So, 0.5 means much worse quality of reconstructed data after decompression. However 0.5 will make the compression have a higher compression ratio than 0.005 does. You asked a question regarding the basic concept about error bound. You can check this paper for more details: Dingwen Tao, Sheng Di, Hanqi Guo, Zizhong Chen, and Franck Cappello, "Z-checker: A Framework for Assessing Lossy Compression of Scientific Data", in The International Journal of High Performance Computing Applications (IJHPCA), 2017

mearcla commented 2 months ago

I have another question please, how can I get on the size of uncompressed data?

float uncompdata = (float) SZ_decompress(SZ_FLOAT, bytes, byteLength, r5, r4, r3, r2, r1);

disheng222 commented 2 months ago

The size of uncompressed data should be the same as that of the original dataset. It's preknown. Or, you can compute it based on the dimensions, r5, r4, ..., r1.

Best, Sheng

On Tue, Aug 20, 2024 at 4:44 AM mearcla @.***> wrote:

I have another question please, how can I get on the size of uncompressed data?

float uncompdata = (float) SZ_decompress(SZ_FLOAT, bytes, byteLength, r5, r4, r3, r2, r1);

— Reply to this email directly, view it on GitHub https://github.com/szcompressor/SZ/issues/118#issuecomment-2298434260, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACK3KSINQ2S4YZ5UBKMWV6LZSMFYNAVCNFSM6AAAAABMOC7BF2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJYGQZTIMRWGA . You are receiving this because you commented.Message ID: @.***>