search

ESMValGroup / ESMValTool

ESMValTool: A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
https://www.esmvaltool.org
Apache License 2.0
216 stars 126 forks source link

Derived ocnBgchem variables can't find FX files on jasmin #773

Closed ledm closed 5 years ago

ledm commented 5 years ago

I'm attempting to test the ocean biogeochemistry recipe on jasmin and I've encountered the following bug when evaluating the derived variable gtfgco2. I believe that this bug arises due to the absence of ocnBgchem-specific FX files. However, this is not a problem that appears with normal (non-derived) variables in the same recipe.

Note that I had not encounter this problem on my local machine, as the path to the FX files was much less complex. This is so far only a problem for me on Jasmin.

When calculating the derived variable, gtfgco2, ESMValTool looks for the FX files in the following location on jasmin:

/badc/cmip5/data/cmip5/output1/*/*/historical/fx/ocnBgchem/fx/r0i0p0/latest/areacello

However, there is no ocnBgchem folder, as the FX files only exist in the atmos, land, & ocean directories (for most cmip5 models). For example, see:

/badc/cmip5/data/cmip5/output1/*/*/historical/fx/ocean/fx/r0i0p0/latest/areacello

Is there a way to explicitly require the derived variable to use the ocean FX files if the ocnBgchem FX files are not available?

I imagine that similar problems may arise for derived variables in atmospheric chemistry, sea ice, as well as ocean biogeochemistry, as many models do not always include FX files.

valeriupredoi commented 5 years ago

this https://github.com/ESMValGroup/ESMValTool/pull/763/commits/a8d25ae732de57b561057dc705c58a96130ddcae should fix this issue - @bouweandela you owe me a beer since this used to work before the _data_finder re-haul :grin:

bouweandela commented 5 years ago

To me it looks like this problem is not related to derived variables, because you are unable to find the fx file for the normal variable fgco2, which is used to derive gtfgco2. Apparently it is not stored in the realm listed with the variable in the cmor table.

valeriupredoi commented 5 years ago

Yes exactly, I patched this on the PR, should be fine now, at least for this corner case

Dr Valeriu Predoi. Computational scientist NCAS-CMS University of Reading Department of Meteorology Reading RG6 6BB United Kingdom

On Thu, 10 Jan 2019, 14:38 Bouwe Andela <notifications@github.com wrote:

To me it looks like this problem is not related to derived variables, because you are unable to find the fx file for the normal variable fgco2, which is used to derive gtfgco2. Apparently it is not stored in the realm listed with the variable in the cmor table.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ESMValGroup/ESMValTool/issues/773#issuecomment-453098439, or mute the thread https://github.com/notifications/unsubscribe-auth/AbpCo37PIz7r7EZAov5mzzxuylK1OGzlks5vB0JNgaJpZM4ZaH8e .

ledm commented 5 years ago

Apologies for the delay, but I'm pretty sure that this patch has worked.

I've run it over the gtfgco2 variable by itself, and now I'm checking the entire recipe_ocean_bgc.yml recipe to make sure.

valeriupredoi commented 5 years ago

@ledm can we close this?

ledm commented 5 years ago

yep!

bouweandela commented 5 years ago

I recommend not closing issues until the PR in which they are solved is actually merged.