JPLMLIA / pdsc

Planetary Data System Coincidences
https://jplmlia.github.io/pdsc/
Other
3 stars 2 forks source link

Reduce memory footprint of ingestion process #32

Open garydoranjr opened 4 years ago

garydoranjr commented 4 years ago

For HiRISE EDRs, ingesting the cumulative index can take O(10 GB) of memory. One cause could be the construction of many segment objects prior to the index being created. Investigate using __slots__ to reduce the memory footprint of these objects.