search

shuzhao-li-lab / asari

asari, metabolomics data preprocessing
Other
38 stars 9 forks source link

Pickle Alternative #73

Closed jmmitc06 closed 2 weeks ago

jmmitc06 commented 1 year ago

Pickle for processing intermediates, e.g., extracted mass tracks is probably fine, pickle probably should not be used when moving data to a new process in the interest of safety. JSON is a decent alternative.

jmmitc06 commented 2 weeks ago

With the newest addition to the compress_tracks branch this is largely solved when 'hybrid' mode is used. While hybrid mode still uses compressed pickle, its never saved to disk and thus cannot be loaded by a different process.