ESMValTool as a validation tool for EcoExtreML

[yijianzeng]

From Bob:

• I think we need to perform a station-based validation of the different datasets before deciding which one should be used for global simulation. We may need to do some CDF matching or copulas matching as Sarah did for the ERA reanalysis data. These are valid for meteorological data, CO2 data and radiation data. It seems CAMS reanalysis contain all these data but we need to have a first validation. An idea is is extract all the relevant data and compare them to GEWEX PLUMER2 in-situ data.
• Yijian: we may see if we should use PLUBMER2 metrics or the C3S one (or even we have have proposed in CORE-CLIMAX). Bart/Sarah: Could one of the eScience existing validation tool be used for this purpose? so that we can do automatic quality control.

[SarahAlidoost]

We can explore whether the support is already there or how we can contribute to esmvaltool. This is a list of datasets and this is a list of metrics supported by ESMValTool. Here is an example to compare precipitation from three different resources done via ESMValTool.

[SarahAlidoost]

This is a list of recipes (workflows) in esmvaltool. The list is organized into categories like climate, land, .... Some of the analysis done in the recipes might be close to what you suggested here and then can be reused.

[yijianzeng]

Thanks a lot, Sarah, i think there are several existing receipts for land that can be used for STEMMUS-SCOPE. Particularly the following:

• Landcover - Albedo
• Hydrological models - data pre-processing
• Hydro forcing comparison
• Landcover diagnostics
• Land-surface Surface Radiation - Autoassess diagnostics

These receipts will be valuable inputs! + Yunfei's post-processing (analysis) scripts = STEMMUS-SCOPE Postprocessing Module for the global platform.

[yijianzeng]

@SarahAlidoost @bobzsu The backend of the analysis engine (for modelevaluation.org) is the Data conversion and analysis scripts for the PALS (Protocol for the Analysis of Land Surface models):

https://github.com/aukkola/palsR

And below are Scripts to create the forcing and evaluation dataset for the PLUMBER2 ecosystem model intercomparison project: https://github.com/aukkola/PLUMBER2

[bobzsu]

https://modelevaluation.org/analyses/workspace/

I checked the evaluation for STEMMUS-SCOPE results. I think it is sufficient to make the same comparison for each driving variable in a similar manner: i.e. plot time series, Taylor diagram, scatter plot, pdf, annual cycle, diurnal cycle and the summary table of statistics.

We need to see if the selected forcing variables are consistent with in-situ observations (compare to PLUMBER2 forcing in this case). If a variable is not consistent, we shall then discuss what to do.