search

mrkkrp / zip

Efficient library for manipulating zip archives
Other
81 stars 27 forks source link

Excess memory usage #80

Open tfc opened 3 years ago

tfc commented 3 years ago

Hi there,

[edit: i reduced the app and added some heap profile graphs]

i wrote a little example programm (see my branch to try it out: https://github.com/tfc/zip)

import Codec.Archive.Zip

main :: IO ()
main = withArchive "archive.zip" $ unpackInto "output_dir"

archive.zip is just a ~40MB nixpkgs zip-ball (see my repo link: test.sh downloads this file for anyone who wants to try it at home).

different profiles:

unziptest-hc

unziptest-hd

unziptest-hm

unziptest-hr

unziptest-hy

so the library consumes ~200 mb ram for unpacking a 40mb zip archive and i think this should be less, right? From looking at the different profiles, it seems that the String type of the zip entries blows up quite a bit, might that be?

mrkkrp commented 3 years ago

Hi, thanks for the report. I will look at this eventually, but I can't promise anything in the coming 2 weeks. If you figure it out yourself please feel free to comment here and/or open a PR.