search

ESMValGroup / ESMValTool

ESMValTool: A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
https://www.esmvaltool.org
Apache License 2.0
223 stars 128 forks source link

CMUG WP5.4 CCI LST with Uncertainty propagation #1945

Open morobking opened 3 years ago

morobking commented 3 years ago

Following the work from CMUG WP5.3 to create a diagnostic with CCI LST data, issue #1896 and branch ESACCI-LST, here we look to include the uncertainty information that the CCI files include.

As before, the main metric will be monthly LST created from the daytime and night time overpass files of Aqua MODIS level 3 data, to be compared to CMIP model. An region is defined in the recipe (by the user) and the LST is averaged over this region to create a time series of LST. We will propagate the uncertainty values give in the CCI file to give a total uncertainty value for each time. The CMIP models are then compared on a plot to the CCI LST with this total uncertainty.

Branch created: CMUG-WP5.4-LST

valeriupredoi commented 3 years ago

good stuff @morobking - pls let me and/or @axel-lauer know if any questions! :beer:

morobking commented 3 years ago

@valeriupredoi Moving the conversation here from emails I have changed the Cmorizer to return individual files for each day and night variable, ie tsDay and tsNight. These tables for a custom CMOR table on the ESMValCore branch ESACCI_LST_uncerts When I run ESMValTool I get this error message: esmvalcore._recipe_checks.RecipeError: Could not create all tasks following this message: ERROR Unable to load CMOR table (project) 'OBS' for variable 'tsUnCorErrNight' with mip 'Amon'

Is there something else I need to do get the recipe to see the cmor definitions and hence load the data files?

morobking commented 3 years ago

@valeriupredoi @axel-lauer I'm still having issues getting ESMValTool to find the new variables when running a recipe. This has worked ok for the cmorizer, but not for the next part. I'm still getting this message: ERROR Unable to load CMOR table (project) 'OBS' for variable 'tsUnCorErrNight' with mip 'Amon'

axel-lauer commented 3 years ago

@morobking Theoretically, it should work if you are on the correct branch for ESMValCore (ESACCI_LST_uncerts) and if you installed everything in "developer" mode (pip install --editable '.[develop]'). Did you do that? @valeriupredoi could you maybe take a brief look?

morobking commented 3 years ago

I've tracked this down to a path issue which is now causing a different issue.

My cmorizer creates individual files for each variable, one file for each monthly timestep. So each netcdf file has one timestep for one variable. I get this error: esmvalcore.cmor.check.CMORCheckError: There were errors in variable lst_unc_loc_atm: time: Frequency not supported by checker in cube: uncertainty from locally correlated errors on atmospheric scales / (kelvin) (time: 1; latitude: 3600; longitude: 7200) Dimension coordinates: time x - - latitude - x - longitude - - x Attributes: Conventions: CF-1.7 source_file: /work/scratch-nopw/morobking/OBS/Tier2/ESACCI_LST_UNCERTS/OBS_ESACCI_L... var: tsLocalAtmErrDay

This looks like it comes from the recipe not the diagnostic. I'm sure the diagnostic isnt being run yet. But this started when I got the issue I previously mentioned sorted. Is this still a path issue and ESMValTool isn't getting the right custom CMOR tables? Or something else?

axel-lauer commented 3 years ago

@morobking I tried to reproduce this error on Mistral (DKRZ) but had no success. Instead, the preprocessor successfully produced output for variables tsLocalSfcErrDay and tsUnCorErrNight. Here is what I did:

The ESMValTool preprocessor finishes successfully, but then the script lst_uncert/lst_uncert.py crashes with division by zero. I did get some warnings about the longname being changed for variable tsLocalSfcErrDay. This is probably because the cmorized ESACCI data look a bit strange, i.e. the original variable name and longname are being kept instead of being replaced by the information from the corresponding custom cmor tables. Not sure why, maybe @valeriupredoi has an idea?

Anyway, since the preprocessor successfully produces some output for the uncertainty variables, I would guess your problem might be caused by e.g. ESMValCore not being installed in 'develop mode', the wrong ESMValCore branch being checked out or possible a broken or incompatible conda environment. @valeriupredoi we might need your help here...

valeriupredoi commented 3 years ago

cheers @axel-lauer for testing :+1: @morobking can you please post the output of conda list esmvalcore && conda list esmvaltool && which esmvaltool - suspecting Axel is quite right, must be a mismatch between the needed dev Core branch and the actual use of either main or the released version - easy fix at any rate :+1:

morobking commented 3 years ago

@axel-lauer @valeriupredoi Thank you both you looking at this. Just to say the print(0/0) is deliberate, I know its a lazy way of stopping the code while I'm still developing it.

I do have a path in my config yml file that says: config_developer_file: /home/users/robking/ESMValTool/config-developer.yml I think I needed this for the cmorizer to find something, but I honestly forget who told me that. Or if its true.

Running the request from Valeriu: conda list esmvalcore && conda list esmvaltool && which esmvaltool # packages in environment at /home/users/robking/miniconda3/envs/esmvaltool: # # Name Version Build Channel esmvalcore 2.2.0 dev_0 # packages in environment at /home/users/robking/miniconda3/envs/esmvaltool: # # Name Version Build Channel esmvaltool 2.2.0 dev_0 esmvaltool-sample-data 0.0.3 pypi_0 pypi ~/miniconda3/envs/esmvaltool/bin/esmvaltool

morobking commented 3 years ago

@valeriupredoi Is there a way to fix those errors please?

valeriupredoi commented 3 years ago

Hi Rob,

I have opened two pull requests - one in Core https://github.com/ESMValGroup/ESMValCore/pull/1308 and another in Tool https://github.com/ESMValGroup/ESMValTool/pull/2291 - this way keep things tidy and organized, and allow for multiple pairs of eyes to see and comment on the code and functionalities. Could you please have a look at this comment https://github.com/ESMValGroup/ESMValCore/pull/1308#issuecomment-915100713 and follow those steps to have both your latest branches installed in development mode and repeat your test runs? If there are any issues, I'd suggest posting straight on the relevant PR page so we keep things visible and - what do you know - you may not have to wait 3 weeks for a reply from me since I totally buried the email :grin: