SIO-ODF / ctdcal

A CTD calibration package in use on US GO-SHIP cruises
BSD 3-Clause "New" or "Revised" License
11 stars 5 forks source link

Consolidated manual flagging #106

Closed asmith75218 closed 3 months ago

asmith75218 commented 3 months ago

Consolidates manual flagging of bottle data.

Manual flags are recorded in a new custom format grouped by parameter with associated reference data, referred to as a node in the data format, and stored to a single JSON file. Flag data are loaded as a Munch object, a subclass of Python dictionary, allowing attribute-style access and convenience methods for converting to and from JSON. Additional convenience functions are provided to convert between Pandas DataFrames to support existing workflow.

The data format is unconstrained to allow flexibility and to support reference parameters that may be implemented in the future, or parameters from multiple rosettes.

Some changes to _oxyfitting, _processctd and rinko are included only as needed to update deprecated code and resolve errors that prevent the execution of the program.

The project requirements are updated to include Munch.

asmith75218 commented 3 months ago

Good changes. Looking forward to the final beta release and running through test data to check node functionality prior to the refactor.

I'm pretty confident in regular usage, but we'll definitely want to watch for edge cases. Flagging of asterisk lines in _odfio identified a few gotchas, but this was largely because existing modules each handle flagging a little (or a lot) differently. As we move forward, we can make this be consistent across modules, and ensure reliability.

QC tool is improved here but might not be as good as it could be, keeping that issue open but I'm going to mention this PR there.

The only changes I made were transparent loading/saving of flags, and a couple minor changes for compatibility with more recent Bokeh versions. The tool still needs all the improvements, especially considering it is the only way to input manual flags at the moment. This version only supports ODF-style salts.