processing a file should not load the entire file in memory when processing

tagbase / tagbase-server

tagbase-server is a data management web service for working with eTUFF and nc-eTAG files.

https://oiip.jpl.nasa.gov/doc/OIIP_Deliverable7.4_TagbasePostgreSQLeTUFF_UserGuide.pdf

Apache License 2.0

7 stars 2 forks source link

processing a file should not load the entire file in memory when processing #240

Closed renato2099 closed 1 year ago

renato2099 commented 1 year ago

Processing an observations file shouldn't imply loading the entire file a-priori when processing it. We should process it as we read it to reduce memory usage