buglabs / node-xml2json

Converts XML to JSON using node-expat
808 stars 209 forks source link

There are errors in your xml file: out of memory #151

Open TianboJi opened 6 years ago

TianboJi commented 6 years ago

I have a 1.7GB xml file which is exported from FileMaker Pro. When I use this module to parse it, it returns

Error: There are errors in your xml file: out of memory

Is there any way to solve it? I already add the parameter "--max-old-space-size=4096" but it doesn't work

aakashmalhotra commented 6 years ago

This plugin does not stream the file so the whole file is loaded into memory. Hence, the error you have. Solution: Stream your data

c4milo commented 6 years ago

@aakashmalhotra is right, even though we use node-expat which parses the XML document in streaming, we don't stream the resulting JSON structure out of the function.

Matthbo commented 6 years ago

Isn't it a good idea to add an option to make it a stream?

c4milo commented 6 years ago

@matthbo, yes, it would be useful with big documents. I just don’t currently have the bandwidth to do it :/. Although, I’m open to reviewing contributions!

cangSDARM commented 4 years ago

I'm handling a 8G xml file. It's have the same problem for me. Hope it support streamable API in 2021

aakashmalhotra commented 4 years ago

Hi @cangSDARM,

you can use if you are dealing with huge files https://github.com/astro/node-expat

cangSDARM commented 4 years ago

@aakashmalhotra Thanks. I'll check that.