Compatibility with other zlib implementations?

[IsoLinearCHiP]

Hi,

I am trying to make/find a python module to interoperate with your LibDeflate implementation.

I've tried using python 3.11 stdlib zlib library with no success.

I tried the following snippet in python to generate a compressed file:

  import zlib
  import os

  test_str = b'aaaa'
  compressed = zlib.compress(test_str, wbits=-9)
  with open('test_py.z', 'wb') as f:
      f.write(compressed)

And then tried to decompress it with lua .\LibDeflate.lua -d test_py.z test_py.txt, but only got the error:

LibDeflate: Decompress fails.

If I change the test_str to b'aa' I indeed get Successfully writes 2 bytes, so far that's all I have managed.

I also tried twiddling around with the wbits parameter from https://docs.python.org/3/library/zlib.html#zlib.compress but neither 9, -9, 25 nor leaving it out made any difference.

I also tried using LibDeflate:DecompressDeflate() but so far I have again only been able to transfer the string 'aa' and nothing else.

Would you be able to point me in the right direction?

[IsoLinearCHiP]

As a followup. The oposite direction seems to be working better.

Wrote a few bytes to a file.

with open('test.txt', 'wb') as f:
    f.write(b'Hello World! This is a test of compression with LibDeflate.lua')

Ran it through LibDeflate

lua .\LibDeflate.lua test.txt test.z

And then opened it up in python again:

  import zlib

  with open('test.z', 'rb') as f:
      compressed = f.read()
  decompressed = zlib.decompress(compressed, wbits=-8)
  print(decompressed)

b'Hello World! This is a test of compression with LibDeflate.lua'

[mahmoudimus]

Did you figure this out @IsoLinearCHiP ? Otherwise, I will port it over when I have some time.

[whi-tw]

Ah excellent, others with the same questions!

I started work porting this, but realised that for what I wanted, it was overkill, so I'm using the actual lua library via lupa for now.

My 'project': https://github.com/whi-tw/wow-archon-itl-exporter

I will get around to porting eventually though. Maybe.

[mahmoudimus]

I found out how to do this btw:

After reading RFC 1950, I was able to understand that a zlib stream is constructed with python's zlib library. This stream is prefixed with a 2-byte header (0x78, 0x9C) and is suffixed via the Adler-32 checksum of the uncompressed data (which equals 4 bytes).

So, to get it to work in Python:

import zlib
bytestring = b"hello"
compressed = zlib.compress(bytestring)
print('compressed is: ', compressed.hex())
with open('bs.z', 'wb') as f:
    f.write(compressed[2:-4])

then, using lua:

lua .\LibDeflate.lua -d bs.z decompressed.txt

which works :)

for decompression of a lua deflated string, just use zlib.decompress(bytestring, wbits=-8)

Hope this helps.