search

prewk / xml-string-streamer

Stream large XML files with low memory consumption.
MIT License
356 stars 49 forks source link

prevent memory leak using the unique node parser #64

Closed Nemo64 closed 5 years ago

Nemo64 commented 5 years ago

after the first node has been found, the unique node parser keeps everything in memory until it finds the next node.

If there is a node at the beginning of the file and then no more, this will result in out of memory issues.

I extended the discard logic to always discard the old string while searching for the next opening node.

prewk commented 5 years ago

Thanks for you contribution!

It's been a long while since I worked on this package, so my memory is a bit bad, but it doesn't break any tests so it looks okay.

I'll have a closer look :)

prewk commented 5 years ago

Published 0.11.1 with the fix in it. Thanks again.

Nemo64 commented 5 years ago

Great that you still support this package.

I think there is something wrong with the packagist auto-update, the new version didn't come though. https://packagist.org/packages/prewk/xml-string-streamer

prewk commented 5 years ago

Oh, apparently they deprecated something in 2019. I'm gonna look into it. Thanks.

prewk commented 5 years ago

There we go :)

Nemo64 commented 5 years ago

awesome, thanks.