inflate fails with guard stream.dst_size == 0 returns nil

[RTSMikeM]

I'm bring back some binary data from a web server using URLSession and POST.

let (data, response) = try await m_oURLSession.data(for: request)

after checking the content-encoding header is "deflate"

I run this:

inflatedData = data.inflate()

Which fails down deep and returns nil because stream.dst_size is not zero:

switch compression_stream_process(&stream, flags) { case COMPRESSION_STATUS_OK: guard stream.dst_size == 0 else

I know the data is actually deflate because I've got an Android app which processes the same deflated data successfully. Any ideas on what might be happening here? Am I missing some conversion of data before calling inflate ? TY

[RTSMikeM]

Also I've used zlib a lot in C++ with deflate.

It looks to me like the call to perform should be be inside a loop. Otherwise you can exhaust your output buffer (or your input bufffer). I've got 17,xxxx bytes coming in that will inflate to something rather large.

Also it appears you are trying to finalize on the 1st pass. You cannot finalize until you've hit COMPRESSION_STATUS_END and no more output will come. (this is what I can glean from my usage of zlib in C++)

Have you tried this code on a set of bytes?

[mw99]

Hi.

Contrary to popular believe, when a HTTP server tell you it sends deflate, it actually sends you deflated data wrapped with a zlib header and footer. Please see https://www.rfc-editor.org/rfc/rfc2616#section-3.5

deflate The "zlib" format defined in RFC 1950 [31] in combination with the "deflate" compression mechanism described in RFC 1951 [29].

Therefore, please change

inflatedData = data.inflate()

to

inflatedData = data.unzip()

and your stuff should be working.

---

In regards to android: Sometimes there are HTTP server implementations that actually send you deflated data because, I guess they goy the RFC wrong, and android covers for these wrong servers I guess. My library obviously doesn't do that.

---

In regards to your code critic, I can't really follow you there. Have you seen my unit tests? This code is working for thousands of people. If you find an error in the code, please send a pull request including test data and/or unit tests that fail.

Otherwise you can exhaust your output buffer

This is not s steaming library. This is more of an easy to use library to decompress "small" stuff. (under 20MB I guess)

At one point in time, all compressed data, and decompressed data will be both loaded into memory. If that is too much for you, you will have to find another solution that provides you the data in chunks.

[RTSMikeM]

It's not zip. I did try your suggestion and it fails. It is definitely raw deflate. I know this because we wrote it. :-)

Here's how the inflate is working in android:

InflaterInputStream in = new InflaterInputStream(m_oUrlConnection.getInputStream(), new Inflater(true)); iBytesRead = in.read(arrBytes, 0, 1024); while (-1 != iBytesRead) { oOutStream.write(arrBytes, 0, iBytesRead); iBytesRead = in.read(arrBytes, 0, 1024); }

Nothing special there.

[RTSMikeM]

I was able to save the data, only 4K in size, to a file after it was deflated raw. I'm going to try to load it directly via a test harness and see if it inflates. I can send it to you if you wish. Thanks for the help!

[RTSMikeM]

I've got the deflate file on my Mac desktop. How can I load on a device for testing? I'm very new to iOS. I'm using an iPod touch device or simulator for testing.

[RTSMikeM]

I figured it out! let (data, response) = try await m_oURLSession.data(for: request)

The data coming back from URLSession is already inflated automatically by the system . FYI your code works beautifully. I was able to read a file from my apps bundle and inflated it perfectly. I saw the byte count was the same as what was coming back from the URLSession. That's how I figured it out.

Thanks! Close this issue please for me as resolved.

[RTSMikeM]

Closed