British-Oceanographic-Data-Centre / COAsT

A Coastal Ocean Assessment Tool built as an extendable python framework for nemo models
https://british-oceanographic-data-centre.github.io/COAsT/
MIT License
24 stars 11 forks source link

Add renaming of domain variables according to mapping in config file #503

Open jasontempestholt opened 2 years ago

jasontempestholt commented 2 years ago

This is to allow different names for bathymetry variable. Needs to happen in load_domain, before bathyemtry is set. Tried adding: for key, value in self.config.domain.variable_map.items(): try: dataset_domain = dataset_domain.rename_vars({key: value})0 except ValueError as err: etc..

but this breaks dataset_set_coord()

jpolton commented 2 years ago

Story:

If a domain_cfg.nc file has a bathymetry variable, on loading this variable can not be found unless it has the hardwired expected name. Furthermore the bathymetry variable name can not be changed.

AC: Want to be able to apply variable name remapping (via json file) to a bathymetry variable in domain_cfg.nc

lukegorman commented 2 years ago

we need this domain_cfg.nc file please.

and if you are so inclined, a full example of how to reproduce

jpolton commented 2 years ago

OK so I am not sure that this is much improved on what you have already managed to achieve:

import coast
import matplotlib.pyplot as plt
import datetime
import numpy as np

Create new domain file swapping bathy_metry for bathy_name test_bathy_domain.nc : https://www.dropbox.com/s/gq3uoxubefmkcbk/test_bathy_domain.nc?dl=0 (40Mb)

Create a new json file, attempting to map bathy_name to bathy_metry. Put it in config/ example_nemo_grid_t_NEW.json : https://www.dropbox.com/s/g07jwfkrxfj9tn5/example_nemo_grid_t_NEW.json?dl=0

fn_nemo_dat = "./example_files/coast_example_nemo_data.nc"
fn_nemo_dom_NEW = "./example_files/test_bathy_domain.nc"
fn_nemo_dom_OLD = "./example_files/coast_example_nemo_domain.nc"
fn_config_t_grid_OLD = "./config/example_nemo_grid_t.json"
fn_config_t_grid_NEW = "./config/example_nemo_grid_t_NEW.json"

Check the old stuff works:

nemo_OLD = coast.Gridded(fn_data = fn_nemo_dat, fn_domain = fn_nemo_dom_OLD, config=fn_config_t_grid_OLD)

Loads fine. The bathymetry variable exists and is not zeros nemo_OLD.dataset.bathymetry.argmax().values > 10 is True

Try with the new domain file:

nemo_NEW = coast.Gridded(fn_data = fn_nemo_dat, fn_domain = fn_nemo_dom_NEW, config=fn_config_t_grid_OLD)
/Users/jeff/GitHub/COAsT/coast/data/gridded.py:182: UserWarning: The model domain loaded, './example_files/test_bathy_domain.nc', does not contain the bathy_metry' variable. This will result in the NEMO.dataset.bathymetry variable being set to zero, which may result in unexpected behaviour from routines that require this variable.
  warnings.warn(

Missing hardwired, expected, variable

Can we fiddle with the json file to map bathy_name to bathy_metry? Try with the new config file

nemo_NEW2 = coast.Gridded(fn_data = fn_nemo_dat, fn_domain = fn_nemo_dom_NEW, config=fn_config_t_grid_NEW)
/Users/jeff/GitHub/COAsT/coast/data/gridded.py:182: UserWarning: The model domain loaded, './example_files/test_bathy_domain.nc', does not contain the bathy_metry' variable. This will result in the NEMO.dataset.bathymetry variable being set to zero, which may result in unexpected behaviour from routines that require this variable.
  warnings.warn(

So a bathymetry variable is created but it is empty.

nemo_NEW2.dataset.bathymetry.argmax().values > 10 is FALSE

jpolton commented 2 years ago

FYI A quirk of the gridded.py coding is that the expected bathymetry variable is bathy_metry. If this doesn't exist it creates a bathymetry variable. The final version is stored as self.dataset.bathymetry and bathy_metry is discarded. Maybe this was all self apparent. I am just rediscovering it.

roje-bodc commented 2 years ago

@jpolton we believe we've got it working on our new branch at the moment. The json config file was slightly incorrect as the variable was named 'bathy_name', but in the file it is 'bath_name'.

We've created a pull request with the changes: https://github.com/British-Oceanographic-Data-Centre/COAsT/pull/519