search

ACCESS-NRI / ESMValTool-workflow

2 stars 1 forks source link

readme

ACCESS-NRI ESMValTool-workflow

ACCESS-NRI maintenance of ESMValTool for the Australian Community.

What is ESMValTool?

The Earth System Model Evaluation Tool (ESMValTool) is a tool developed for evaluation of Earth System Models in CMIP (Climate Model Intercomparison Projects). It allows for routine comparison of single or multiple models, either against predecessor versions or against observations. ESMValTool is a community-developed climate model diagnostics and evaluation software package, driven both by computational performance and scientific accuracy and reproducibility. It is open to both users and developers, encouraging open exchange of diagnostic source code and evaluation results from the Coupled Model Intercomparison Project CMIP ensemble.

What are we releasing?

ACCESS-NRI is releasing an NCI configuration of ESMValTool under the name ESMValTool-workflow.

ESMValTool-workflow is the ACCESS-NRI software and data infrastructure that enables the ESMValTool evaluation framework on NCI Gadi. It includes the ESMValTool/ESMValCore Python packages, the ESMValTool collection of recipes and diagnostics and some observational datasets. ESMValTool-workflow is configured to use the existing NCI supported CMIP data collections.

Using ESMValTool on Gadi

ESMValTool is provided through the xp65 project on Gadi.

Pre-requisites

To enable the ESMValTool-workflow, you need to be a member of the xp65 NCI projects:

Depending on your needs, you may want to also joined the following supported data collections:

Loading the ESMValTool-workflow modules

To load the the esmvaltool module, execute the following commands:

    module use /g/data/xp65/public/modules
    module load esmvaltool

ESMValTool is pre-configured to access CMIP and observation datasets available on Gadi. By default, ESMValTool looks for the config_user.yml file in the home directory, inside the .esmvaltool folder. You can get a copy by running:

esmvaltool config get_config_user --path=dest

To list which ESMValTool recipes are available on Gadi, run:

esmvaltool recipes list

To find out details of a specific recipe_name.yml, execute:

esmvaltool recipes show recipe_name.yml

To retrieve a recipe (and modify it), execute:

esmvaltool recipes get recipe_name.yml

To execute recipe_name.yml and automatically download the required climate data to the default directory, run:

esmvaltool run examples/recipe_python.yml --search_esgf=when_missing

The --search_esgf=when_missing option tells ESMValTool to search for and download the necessary climate data files from Earth System Grid Federation (ESGF), if they cannot be found locally.

Recipes current status

Recipes without observation datasets

Name status
recipe_combined_indices recipe_combined_indices
recipe_modes_of_variability recipe_modes_of_variability
recipe_li17natcc recipe_li17natcc
recipe_extreme_index recipe_extreme_index
recipe_multimodel_products recipe_multimodel_products
recipe_seaice_feedback recipe_seaice_feedback
recipe_hyint_extreme_events recipe_hyint_extreme_events
recipe_capacity_factor recipe_capacity_factor
recipe_ocean_scalar_fields recipe_ocean_scalar_fields
recipe_tebaldi21esd recipe_tebaldi21esd
recipe_psyplot recipe_psyplot
recipe_climate_change_hotspot recipe_climate_change_hotspot
recipe_ocean_amoc recipe_ocean_amoc
recipe_russell18jgr recipe_russell18jgr
recipe_diurnal_temperature_index recipe_diurnal_temperature_index
recipe_seaborn recipe_seaborn
recipe_ensclus recipe_ensclus
recipe_zmnam recipe_zmnam
recipe_cvdp recipe_cvdp
recipe_consecdrydays recipe_consecdrydays
recipe_kcs recipe_kcs
recipe_autoassess_landsurface_permafrost recipe_autoassess_landsurface_permafrost
recipe_carvalhais14nat recipe_carvalhais14nat
recipe_rainfarm recipe_rainfarm
recipe_eady_growth_rate recipe_eady_growth_rate
recipe_toymodel recipe_toymodel
recipe_williams09climdyn_CREM recipe_williams09climdyn_CREM
recipe_heatwaves_coldwaves recipe_heatwaves_coldwaves
recipe_thermodyn_diagtool recipe_thermodyn_diagtool
recipe_ecs recipe_ecs
recipe_runoff_et recipe_runoff_et
recipe_tcr recipe_tcr
recipe_hyint recipe_hyint
recipe_ocean_ice_extent recipe_ocean_ice_extent
recipe_ocean_example recipe_ocean_example
recipe_meehl20sciadv recipe_meehl20sciadv
recipe_deangelis15nat_fig1_fast recipe_deangelis15nat_fig1_fast
recipe_python_for_CI recipe_python_for_CI
recipe_flato13ipcc_figure_942 recipe_flato13ipcc_figure_942
recipe_monitor recipe_monitor
recipe_monitor_with_refs recipe_monitor_with_refs
recipe_preprocessor_derive_test recipe_preprocessor_derive_test
recipe_python recipe_python
recipe_preprocessor_test recipe_preprocessor_test
recipe_my_personal_diagnostic recipe_my_personal_diagnostic
recipe_extract_shape recipe_extract_shape
recipe_julia recipe_julia
recipe_r recipe_r
recipe_ncl recipe_ncl

Tier 1 recipes

Name status
recipe_validation recipe_validation
recipe_autoassess_landsurface_surfrad recipe_autoassess_landsurface_surfrad
recipe_ocean_quadmap recipe_ocean_quadmap
recipe_radiation_budget recipe_radiation_budget
recipe_quantilebias recipe_quantilebias
recipe_clouds_ipcc recipe_clouds_ipcc
recipe_bock20jgr_fig_8-10 recipe_bock20jgr_fig_8-10

Tier 2 recipes

Name status
recipe_seaice recipe_seaice
recipe_ocean_bgc recipe_ocean_bgc
recipe_landcover recipe_landcover
recipe_sea_surface_salinity recipe_sea_surface_salinity
recipe_arctic_ocean recipe_arctic_ocean
recipe_cox18nature recipe_cox18nature
recipe_wenzel16nat recipe_wenzel16nat
recipe_collins13ipcc recipe_collins13ipcc
recipe_albedolandcover recipe_albedolandcover
recipe_autoassess_landsurface_soilmoisture recipe_autoassess_landsurface_soilmoisture
recipe_ocean_Landschuetzer2016 recipe_ocean_Landschuetzer2016
recipe_wenzel14jgr recipe_wenzel14jgr
recipe_ocean_multimap recipe_ocean_multimap
recipe_esacci_oc recipe_esacci_oc
recipe_esacci_lst recipe_esacci_lst
recipe_flato13ipcc_figure_98 recipe_flato13ipcc_figure_98
recipe_flato13ipcc_figure_914 recipe_flato13ipcc_figure_914
recipe_flato13ipcc_figure_924 recipe_flato13ipcc_figure_924
recipe_ipccwg1ar6ch3_fig_3_9 recipe_ipccwg1ar6ch3_fig_3_9
recipe_lauer22jclim_fig9-11c_pdf recipe_lauer22jclim_fig9-11c_pdf
recipe_lauer22jclim_fig9-11ab_scatter recipe_lauer22jclim_fig9-11ab_scatter

Tier 3 recipes

Name status
recipe_martin18grl recipe_martin18grl
recipe_seaice_drift recipe_seaice_drift
recipe_smpi_4cds recipe_smpi_4cds
recipe_miles_eof recipe_miles_eof
recipe_ecs_constraints recipe_ecs_constraints
recipe_impact recipe_impact
recipe_shapeselect recipe_shapeselect
recipe_perfmetrics_CMIP5_4cds recipe_perfmetrics_CMIP5_4cds
recipe_autoassess_stratosphere recipe_autoassess_stratosphere
recipe_miles_block recipe_miles_block
recipe_miles_regimes recipe_miles_regimes
recipe_climwip_brunner2019_med recipe_climwip_brunner2019_med
recipe_perfmetrics_CMIP5 recipe_perfmetrics_CMIP5
recipe_climwip_test_basic recipe_climwip_test_basic
recipe_climwip_brunner20esd recipe_climwip_brunner20esd
recipe_wenzel16jclim recipe_wenzel16jclim
recipe_snowalbedo recipe_snowalbedo
recipe_extreme_events recipe_extreme_events
recipe_galytska23jgr recipe_galytska23jgr
recipe_gier2020bg recipe_gier2020bg
recipe_eyring06jgr recipe_eyring06jgr
recipe_schlund20esd recipe_schlund20esd
recipe_deangelis15nat recipe_deangelis15nat
recipe_spei recipe_spei
recipe_climwip_test_performance_sigma recipe_climwip_test_performance_sigma
recipe_smpi recipe_smpi
recipe_ecs_scatter recipe_ecs_scatter
recipe_validation_CMIP6 recipe_validation_CMIP6
recipe_cmug_h2o recipe_cmug_h2o
recipe_perfmetrics_land_CMIP5 recipe_perfmetrics_land_CMIP5
recipe_pv_capacity_factor recipe_pv_capacity_factor
recipe_anav13jclim recipe_anav13jclim
recipe_eyring13jgr_12 recipe_eyring13jgr_12
recipe_flato13ipcc_figures_926_927 recipe_flato13ipcc_figures_926_927
recipe_flato13ipcc_figures_938_941_cmip6 recipe_flato13ipcc_figures_938_941_cmip6
recipe_flato13ipcc_figure_945a recipe_flato13ipcc_figure_945a
recipe_flato13ipcc_figures_938_941_cmip3 recipe_flato13ipcc_figures_938_941_cmip3
recipe_flato13ipcc_figures_92_95 recipe_flato13ipcc_figures_92_95
recipe_weigel21gmd_figures_13_16 recipe_weigel21gmd_figures_13_16
recipe_flato13ipcc_figure_96 recipe_flato13ipcc_figure_96
recipe_ipccwg1ar6ch3_atmosphere recipe_ipccwg1ar6ch3_atmosphere
recipe_ipccwg1ar6ch3_fig_3_42_a recipe_ipccwg1ar6ch3_fig_3_42_a
recipe_ipccwg1ar6ch3_fig_3_19 recipe_ipccwg1ar6ch3_fig_3_19
recipe_ipccwg1ar6ch3_fig_3_43 recipe_ipccwg1ar6ch3_fig_3_43
recipe_ipccwg1ar6ch3_fig_3_42_b recipe_ipccwg1ar6ch3_fig_3_42_b
recipe_mpqb_xch4 recipe_mpqb_xch4
recipe_lauer22jclim_fig2_taylor_amip recipe_lauer22jclim_fig2_taylor_amip
recipe_lauer22jclim_fig6_interannual recipe_lauer22jclim_fig6_interannual
recipe_lauer22jclim_fig1_clim recipe_lauer22jclim_fig1_clim
recipe_lauer22jclim_fig2_taylor recipe_lauer22jclim_fig2_taylor
recipe_lauer22jclim_fig5_lifrac recipe_lauer22jclim_fig5_lifrac
recipe_clouds_bias recipe_clouds_bias
recipe_lauer22jclim_fig3-4_zonal recipe_lauer22jclim_fig3-4_zonal
recipe_lauer22jclim_fig7_seas recipe_lauer22jclim_fig7_seas
recipe_lauer22jclim_fig8_dyn recipe_lauer22jclim_fig8_dyn
recipe_lauer22jclim_fig1_clim_amip recipe_lauer22jclim_fig1_clim_amip
recipe_lauer13jclim recipe_lauer13jclim
recipe_daily_era5 recipe_daily_era5
recipe_era5-land recipe_era5-land
recipe_bock20jgr_fig_6-7 recipe_bock20jgr_fig_6-7
recipe_bock20jgr_fig_1-4 recipe_bock20jgr_fig_1-4
recipe_lisflood recipe_lisflood
recipe_hype recipe_hype
recipe_marrmot recipe_marrmot
recipe_wflow recipe_wflow
recipe_pcrglobwb recipe_pcrglobwb
recipe_hydro_forcing recipe_hydro_forcing
recipe_globwat recipe_globwat
recipe_concatenate_exps recipe_concatenate_exps
recipe_check_obs recipe_check_obs
recipe_variable_groups recipe_variable_groups
recipe_correlation recipe_correlation
recipe_decadal recipe_decadal

Observation and reanalyses data products available

You can request access to the ACCESS-NRI collection. Data is stored in /g/data/ct11 on GADI.

We welcome contributions and feedback through the ACCESS-Hive forum. You can also request help through support.

long_name datasets name
0 Total Alkalinity GLODAP, OceanSODA-ETHZ talk
1 Surface pH OceanSODA-ETHZ phos
2 Surface Downward Mass Flux of Carbon as CO2 [kgC m-2 s-1] GCP2018, GCP2020, Landschuetzer2016, OceanSODA-ETHZ fgco2
3 Total Column Ozone ESACCI-OZONE toz
4 TOA Outgoing Longwave Radiation CERES-EBAF, ESACCI-CLOUD, ISCCP-FH, JRA-25, JRA-55, NCEP-NCAR-R1, NOAA-CIRES-20CR rlut
5 Surface Dissolved Inorganic Carbon Concentration OceanSODA-ETHZ dissicos
6 Surface Upwelling Shortwave Radiation CERES-EBAF, ESACCI-CLOUD, ISCCP-FH rsus
7 Primary Organic Carbon Production by All Types of Phytoplankton Eppley-VGPM-MODIS intpp
8 Grid-Cell Area for Ocean Variables OceanSODA-ETHZ areacello
9 Air Temperature AIRS, AIRS-2-1, BerkeleyEarth, CFSR, CRU, CowtanWay, E-OBS, GHCN-CAMS, GISTEMP, GLODAP, HadCRUT3, HadCRUT4, HadCRUT5, ISCCP-FH, Kadow2020, NCEP-DOE-R2, NCEP-NCAR-R1, NOAAGlobalTemp, OceanSODA-ETHZ, PHC, WFDE5, WOA ta
10 Convective Cloud Area Percentage CALIOP, CALIPSO-GOCCP clc
11 Near-Surface Air Temperature BerkeleyEarth, CFSR, CRU, CowtanWay, E-OBS, GHCN-CAMS, GISTEMP, HadCRUT3, HadCRUT4, HadCRUT5, ISCCP-FH, Kadow2020, NCEP-NCAR-R1, NOAAGlobalTemp, WFDE5 tas
12 Specific Humidity AIRS, AIRS-2-1, HALOE, JRA-25, NCEP-NCAR-R1, NOAA-CIRES-20CR hus
13 Daily Maximum Near-Surface Air Temperature E-OBS, NCEP-NCAR-R1 tasmax
14 Percentage of the Grid Cell Occupied by Land (Including Lakes) BerkeleyEarth sftlf
15 Total Cloud Cover Percentage CALIOP, CloudSat, ESACCI-CLOUD, ISCCP, JRA-25, JRA-55, MODIS, MODIS-1-0, NCEP-DOE-R2, NCEP-NCAR-R1, NOAA-CIRES-20CR, PATMOS-x clt
16 Sea Surface Salinity ESACCI-SEA-SURFACE-SALINITY, WOA sos
17 Surface Downwelling Longwave Radiation CERES-EBAF, ISCCP-FH, JRA-55 rlds
18 Mole Fraction of CH4 ESACCI-GHG, TCOM-CH4 ch4
19 Liquid Water Path ESACCI-CLOUD, MODIS lwp
20 Atmosphere CO2 CT2019, ESRL, Scripps-CO2-KUM co2s
21 Surface Downwelling Shortwave Radiation CERES-EBAF, ISCCP-FH rsds
22 Ambient Aerosol Absorption Optical Thickness at 550nm ESACCI-AEROSOL abs550aer
23 Northward Near-Surface Wind CFSR vas
24 Near-Surface Relative Humidity NCEP-NCAR-R1 hurs
25 Sea Surface Temperature ATSR, HadISST, WOA tos
26 Sea Water Potential Temperature PHC, WOA thetao
27 Ambient Aerosol Optical Thickness at 550nm ESACCI-AEROSOL, MODIS od550aer
28 Ambient Aerosol Optical Depth at 870nm ESACCI-AEROSOL od870aer
29 Northward Wind CFSR, NCEP-NCAR-R1 va
30 Sea Water Salinity CALIPSO-GOCCP, ESACCI-LANDCOVER, ESACCI-SEA-SURFACE-SALINITY, PHC, WOA so
31 Tree Cover Percentage ESACCI-LANDCOVER treeFrac
32 Ice Water Path ESACCI-CLOUD clivi
33 Precipitation CRU, E-OBS, ESACCI-OZONE, GHCN, GPCC, GPCP-SG, ISCCP-FH, JRA-25, JRA-55, NCEP-DOE-R2, NCEP-NCAR-R1, NOAA-CIRES-20CR, PERSIANN-CDR, REGEN, SSMI, SSMI-MERIS, TRMM-L3, WFDE5 pr
34 Surface Downwelling Clear-Sky Longwave Radiation CERES-EBAF, JRA-55 rldscs
35 Surface Downwelling Clear-Sky Shortwave Radiation CERES-EBAF rsdscs
36 Delta CO2 Partial Pressure Landschuetzer2016 dpco2
37 TOA Outgoing Clear-Sky Shortwave Radiation CERES-EBAF, ESACCI-CLOUD, ISCCP-FH, JRA-25, JRA-55, NCEP-NCAR-R1 rsutcs
38 Sea Level Pressure E-OBS, JRA-55, NCEP-NCAR-R1 psl
39 CALIPSO Percentage Cloud Cover CALIPSO-GOCCP clcalipso
40 Surface Aqueous Partial Pressure of CO2 Landschuetzer2016, Landschuetzer2020, OceanSODA-ETHZ spco2
41 Surface Upwelling Clear-Sky Shortwave Radiation CERES-EBAF, ESACCI-CLOUD rsuscs
42 Dissolved Inorganic Carbon Concentration GLODAP, MOBO-DIC_MPIM, OceanSODA-ETHZ dissic
43 Mole Fraction of N2O TCOM-N2O n2o
44 Percentage Cloud Cover CALIOP, CALIPSO-GOCCP, CloudSat, ESACCI-CLOUD, ISCCP, JRA-25, JRA-55, MODIS, MODIS-1-0, NCEP-DOE-R2, NCEP-NCAR-R1, NOAA-CIRES-20CR, PATMOS-x cl
45 Surface Upwelling Longwave Radiation CERES-EBAF, ESACCI-CLOUD, ISCCP-FH rlus
46 Carbon Mass Flux out of Atmosphere Due to Net Biospheric Production on Land [kgC m-2 s-1] GCP2018, GCP2020 nbp
47 Total Dissolved Inorganic Phosphorus Concentration WOA po4
48 TOA Incident Shortwave Radiation CERES-EBAF, ESACCI-CLOUD, ISCCP-FH rsdt
49 Omega (=dp/dt) NCEP-NCAR-R1 wap
50 Dissolved Oxygen Concentration CT2019, ESACCI-GHG, ESRL, GCP2018, GCP2020, Landschuetzer2016, Landschuetzer2020, OceanSODA-ETHZ, Scripps-CO2-KUM, WOA o2
51 Percentage Crop Cover ESACCI-LANDCOVER cropFrac
52 Water Vapor Path ISCCP-FH, JRA-25, NCEP-DOE-R2, NCEP-NCAR-R1, NOAA-CIRES-20CR, SSMI, SSMI-MERIS prw
53 Geopotential Height CFSR, NCEP-NCAR-R1 zg
54 Ambient Fine Aerosol Optical Depth at 550nm ESACCI-AEROSOL od550lt1aer
55 Eastward Wind CFSR, NCEP-NCAR-R1 ua
56 Total Dissolved Inorganic Silicon Concentration CFSR, GLODAP, HadISST, MOBO-DIC_MPIM, OSI-450-nh, OSI-450-sh, OceanSODA-ETHZ, PIOMAS, WOA si
57 Mass Concentration of Total Phytoplankton Expressed as Chlorophyll in Sea Water ESACCI-OC chl
58 Natural Grass Area Percentage ESACCI-LANDCOVER grassFrac
59 Daily Minimum Near-Surface Air Temperature E-OBS, NCEP-NCAR-R1 tasmin
60 Condensed Water Path MODIS, NOAA-CIRES-20CR clwvi
61 TOA Outgoing Clear-Sky Longwave Radiation CERES-EBAF, ESACCI-CLOUD, ISCCP-FH, JRA-25, JRA-55, NCEP-NCAR-R1 rlutcs
62 Bare Soil Percentage Area Coverage ESACCI-LANDCOVER baresoilFrac
63 pH GLODAP, OceanSODA-ETHZ ph
64 Surface Carbonate Ion Concentration OceanSODA-ETHZ co3os
65 Surface Air Pressure CALIPSO-GOCCP, E-OBS, ISCCP-FH, JRA-55, NCEP-NCAR-R1 ps
66 Relative Humidity AIRS-2-0, AIRS-2-1, NCEP-DOE-R2, NCEP-NCAR-R1 hur
67 Surface Temperature CFSR, ESACCI-LST, ESACCI-SST, HadISST, ISCCP-FH, NCEP-NCAR-R1 ts
68 Eastward Near-Surface Wind CFSR uas
69 TOA Outgoing Shortwave Radiation CERES-EBAF, ESACCI-CLOUD, ISCCP-FH, JRA-25, JRA-55, NCEP-NCAR-R1, NOAA-CIRES-20CR rsut
70 Daily-Mean Near-Surface Wind Speed CFSR, NCEP-NCAR-R1 sfcWind
71 Surface Total Alkalinity OceanSODA-ETHZ talkos
72 Percentage Cover by Shrub ESACCI-LANDCOVER shrubFrac