cerebis / bin3C

Extract metagenome-assembled genomes (MAGs) from metagenomic data using Hi-C.
GNU Affero General Public License v3.0
23 stars 7 forks source link

Switch to storing contact map as hd5 #16

Closed cerebis closed 3 years ago

cerebis commented 5 years ago

Currently we just pickle the contact map.

Being largely sparse matrices, when saved these objects are fairly space efficient. This and the convenience of pickle save/load made this ok during development.

However, this also comes with the problem that changes to the class ContactMap can invalidate older saved objects -- even if minor properties change or we move dependent packages etc.

Data is data, so we should just save it as such.

We could begin our old HD5 methods, but whether these are correctly/efficiently storing sparse matrices is yet to be determined.

We should

cerebis commented 3 years ago

Duplicate of #34