CovertLab / vEcoli

Whole cell model of E. coli implemented with Vivarium
https://covertlab.github.io/vEcoli/
MIT License
12 stars 3 forks source link

Use DuckDB for analysis scripts #229

Closed thalassemia closed 4 months ago

thalassemia commented 4 months ago

Fix miscellaneous workflow bugs and use DuckDB instead of Polars for analysis scripts (more robust support for larger-than-RAM workloads and more granular control over resource usage).

Add many helper functions for reading data in analysis scripts in ecoli/library/parquet_emitter.py (primarily read_stacked_columns).

Port new gene heatmaps over to serve as a reference for common DuckDB SQL queries.