SIO-ODF / ctdcal

A CTD calibration package in use on US GO-SHIP cruises
BSD 3-Clause "New" or "Revised" License
9 stars 4 forks source link

Save fit params and develop rerun intercomparison tools #76

Open Klankers opened 3 weeks ago

Klankers commented 3 weeks ago

Beta CTDCAL overwrites everything that gets generated in /logs and /pressure (and maybe more). For that reason, data analysts have historically needed to copy and paste the entire data folder, which leads to lack of disk space and data runs themselves require a lot of individual effort to compare results. To make this better on future analysts (as well as users tweaking their workflows), CTDCAL should support instanced runs and allow comparisons between runs without needing to move data around or rerun any processing steps.

DocOtak commented 2 weeks ago

Check out https://github.com/xarray-contrib/datatree that is currently being integrated into xarray Slides from a presentation here: https://speakerdeck.com/tomnicholas/xarray-datatree-hierarchical-data-structures-for-multi-model-science