search

dylex / zip-stream

Haskell ZIP archive streaming processing using conduit
BSD 3-Clause "New" or "Revised" License
7 stars 8 forks source link

Memory/space leak with performing a stream unzip #8

Closed cjordan closed 5 years ago

cjordan commented 5 years ago

Hi,

Thanks for providing this great library! With the help of a kind soul on reddit, I've been able to perform a basic stream unzip on a local file and over http. Unfortunately, all stream unzips appear to leak memory; the aforementioned person on reddit is thinking that the leak is related to the file sizes in the zips. As my files are quite large, the leak is also large. More details here: https://www.reddit.com/r/haskellquestions/comments/9nyp15/using_zipstream_to_unzip_an_archive_through/

Any help is appreciated!

Also, along with this, to help other conduit-uninitiated users, it would be helpful to include simple examples of using this library. The code in that same reddit thread could be copy and pasted somewhere in this library.

dylex commented 5 years ago

Is this related to PR #7 or is there still an additional leak? I should admit I've only tested the zip side in long-running production servers, not unzipping.

As for examples, I presume you looked at the included command-line programs: https://github.com/dylex/zip-stream/blob/master/cmd/unzip.hs

cjordan commented 5 years ago

Thanks for responding. I managed to somehow miss the included program! And, with this very recent PR, the leak appears to have disappeared. Sorry for the trouble, but thanks for the help!