Craven-Biostat-Lab / morphology-prediction

Predictive modeling of cell morphology: an IGVF project.
MIT License
0 stars 0 forks source link

Process profiles in a memory-efficient manner #2

Closed sverchkov closed 1 year ago

sverchkov commented 1 year ago

All (gene perturbation) profiles won't fit in a single table in memory, workflow needs to be designed with that in mind.

sverchkov commented 1 year ago

Using dask seems to be the solution here.