search

openghg / openghg_inversions

University of Bristol Atmospheric Chemistry Research Group RHIME Inversion code (with openghg dependency)
MIT License
5 stars 0 forks source link

Updates to use Openghg v0.8 (zarr backend) #92

Closed brendan-m-murphy closed 5 months ago

brendan-m-murphy commented 7 months ago

This PR includes updates to make OpenGHG inversions compatible with OpenGHG versions 0.7.1 and 0.8 (using zarr backend). This closes issue #43.

Besides changes to the tests and a few minor changes, the way "merged data" is saved and loaded has been changed significantly. By default, if zarr is available, then merged data will be saved to a zarr store; otherwise, the merged data will be saved as a netCDF file.

Merged data can be loaded from zarr, netCDF, or a pickle file. The function load_merged_data in get_data.py searched for merged data in these formats, in this order, and uses the first compatible option found. (So if merged data is saved under the same name with ".zarr" and ".nc" extension, then ".zarr" will be used if zarr is available, otherwise the netCDF file will be used.)

There isn't an option to specify which format is used, but this could be added later.

This closes issue #38.

Further, the continuous integration (CI) workflow has been updated to use three version of OpenGHG: v0.7.1, v0.8, and the devel branch. This will help ensure some backwards compatibility, as well as warn us about conflicts with the devel branch of OpenGHG. (Currently the versions v0.7.1 and v0.8 are hard-coded, so these will need to be updated as new versions of OpenGHG are released.)

This closes issue #89.

qq23840 commented 5 months ago

Currently running a GSN-HFC-23 inversion with this branch. Have had to make a couple of tweaks locally to deal with a time-slicing issue in openghg itself that @gareth-j and I discussed yesterday (https://github.com/openghg/openghg/issues/970) but outside that, the inversion runs fine so all looks good here