search

elasticio / odata2openapi

OData to OpenAPI Converter
https://www.elastic.io
MIT License
33 stars 23 forks source link

XML parsing is very memory intensive #36

Closed jhorbulyk closed 5 years ago

jhorbulyk commented 5 years ago

Currently, the xml2js library referenced consumes lots of memory when parsing xml. For example the metadata document for a typical MS Dynamics metadata document with a size of 10-20 MB can result in memory usage spikes that end up in the 512 MB to 1 GB range. Updating the xml library may resolve this problem.

zubairov commented 5 years ago

@jhorbulyk updating the library means replacing it with some other library?

jhorbulyk commented 5 years ago

I think the first step would be to update to the latest version of the existing library.

ShkarupaNick commented 5 years ago

3 SP Estimation was made according to next actions : 1) Update version of library 2) Investigation of possible improvements of memory usage

uaArsen commented 5 years ago

Investigations results:

  1. Updating library version will not solve a problem. We already use latest version and their git repository was last time updated 2 years ago.
  2. I found another library xml-js that will partially solve this problem, heap usage will be decreased by 50% approximately.
  3. elasticio/odata2openapi repository does not correspond to our libraries code of conduct. Updating accordingly is out of scope of this issue. I am proposing creating new issue and marking this as blocked.
jhorbulyk commented 5 years ago