search

fpco / streaming-commons

Common lower-level functions needed by various streaming data libraries
MIT License
36 stars 39 forks source link

exposed decompressM and compressM #44

Closed ocramz closed 6 years ago

ocramz commented 6 years ago

This PR exposes the two functions decompress' and compress'defined in test/Data/Streaming/ZlibSpec.hs (without the unsafePerformIO, thus renamed decompressM/compressM).

Rationale:

A safe, streaming interface to zlib would be extremely useful. I understand streaming-commons is meant to be minimal on dependencies, but I'm thinking of writing a high-level wrapper to this library with more idiomatic exception handling based e.g. on exceptions (feedDeflate would then throwM and flush the buffer, rather than returning an error code).

snoyberg commented 6 years ago

Sorry, I'm not understanding the motivation here. If you want to create a streaming interface, why do you need a non-streaming set of functions?

ocramz commented 6 years ago

Unless I grossly misunderstand it, compressM and decompressM work in streaming mode until some exceptional condition is encountered, or the stream terminates. This is when we'd like to either throw an exception or return the payload. For example, when interacting with certain web APIs, the response body may or may not be compressed, according to some remote state, which is a condition I'd like to catch. Along with a few other people, I've been asking to surface this type of interface from zlib for a few months ( https://github.com/haskell/zlib/issues/9 ).

snoyberg commented 6 years ago

Can you clarify what you mean by "streaming mode" then? This function will read the entire result into memory.

ocramz commented 6 years ago

I understand; is this fundamentally incompatible with the goals of the library? Alternatively, could you point me to an exception-safe streaming library I may study ?

snoyberg commented 6 years ago

I think there's an inherent misunderstanding here. This library does provide a streaming zlib interface, without the added function you're referencing. You can wrap it in higher levels like is done with conduit: https://www.stackage.org/haddock/lts-10.3/conduit-extra-1.2.3.2/src/Data.Conduit.Zlib.html

ocramz commented 6 years ago

I see the point now; thank you for the clarifications, I will add this functionality elsewhere.