libnxz / power-gzip

POWER NX zlib compliant library
23 stars 18 forks source link

Variable compressed sized in samples #22

Closed pedromfcarvalho closed 4 years ago

pedromfcarvalho commented 4 years ago

The sample code gzip_nxfht_test in the samples/ directory produces a compressed result that varies in size from one invocation to the other, when applied to alice29.txt, also in samples:

pedromfc@:~/power-gzip/samples$ ./gzip_nxfht_test alice29.txt 
crb 0x10a580300
file alice29.txt read, 152089 bytes
compressed 152089 to 75481 bytes total, crc32=66007dba
pedromfc@:~/power-gzip/samples$ ./gzip_nxfht_test alice29.txt 
crb 0x10eaf0300
file alice29.txt read, 152089 bytes
compressed 152089 to 75543 bytes total, crc32=66007dba

The more common case is 75543 bytes, but it sometimes changes. I don't know if is supposed to be non-deterministic, sometimes the sample will always produce 75543 bytes, but sometimes it produces other sizes.

abalib commented 4 years ago

Expected. A future processor may change this behavior.

tuliom commented 4 years ago

@abalib Do you have any references to this behavior?

abalib commented 4 years ago

@tuliom Unfortunately no. We should make a remark in a README. Something like this Internal state of nx-gzip hardware is randomly initialized which may result in compressed stream sizes slightly different between multiple runs of the same input. Use deflateBound() instead to estimate output buffer sizes instead of calling deflate() twice.

abalib commented 4 years ago

I will reopen when it's time for the next processor