search

turbomam / biosample-xmldb-sqldb

Tools for loading NCBI Biosample into an XML database and then transforming that into a SQL database
MIT License
0 stars 1 forks source link

crashed during pivot_harmonized_attributes.py. Out of RAM (on host?) #23

Closed turbomam closed 6 months ago

turbomam commented 6 months ago 
poetry run python biosample_xmldb_sqldb/pivot_harmonized_attributes.py
accessible_tables = ['all_ncbi_attributes_long', 'non_attribute_metadata']
Processing through row 0 of 549805612 at 2024-02-13T02:52:40.573700
pivoted_data.shape = (71155, 159)
...
Processing through row 33000000 of 549805612 at 2024-02-13T03:25:32.826314
pivoted_data.shape = (2280496, 531)
Processing through row 34000000 of 549805612 at 2024-02-13T03:27:08.118753
make: *** [Makefile:170: postgres-pivot] Killed