search

ClimateImpactLab / downscaleCMIP6

Downscaling & bias correction of CMIP6 tasmin, tasmax, and pr for the R/CIL GDPCIR project
MIT License
136 stars 33 forks source link
climate-data climate-impacts cmip6 downscale downscaling gdpcir

readme

.. raw :: html

<div>
<img src="https://raw.githubusercontent.com/ClimateImpactLab/downscaleCMIP6/master/resources/cil-gdpcir-globe.png" style="width: 30%" align="right">
</div>

========================================================== Global Downscaled Projections for Climate Impacts Research

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.6403794.svg :target: https://doi.org/10.5281/zenodo.6403794

.. image:: https://mybinder.org/badge_logo.svg :target: https://mybinder.org/v2/gh/ClimateImpactLab/downscaleCMIP6-binder-env/main?urlpath=git-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252FClimateImpactLab%252FPlanetaryComputerExamples%26urlpath%3Dlab%252Ftree%252FPlanetaryComputerExamples%252Fdatasets%252Fcil-gdpcir%252FREADME.md%26branch%3Dgdpcir-additional-notebooks

The World Climate Research Programme's 6th Coupled Model Intercomparison Project (CMIP6) <https://www.wcrp-climate.org/wgcm-cmip/wgcm-cmip6>_ represents an enormous advance in the quality, detail, and scope of climate modeling.

The Global Downscaled Projections for Climate Impacts Research dataset makes this modeling more applicable to understanding the impacts of changes in the climate on humans and society with two key developments: trend-preserving bias correction and downscaling. In this dataset, the Climate Impact Lab <https://impactlab.org>_ provides global, daily minimum and maximum air temperature at the surface (tasmin and tasmax) and daily cumulative surface precipitation (pr) corresponding to the CMIP6 historical, ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 scenarios for 25 global climate models on a 1/4-degree regular global grid.

Contents:

Additional links:

.. _Accessing the data:

Accessing the data

GDPCIR data can be accessed on the Microsoft Planetary Computer: planetarycomputer.microsoft.com/dataset/group/cil-gdpcir <https://planetarycomputer.microsoft.com/dataset/group/cil-gdpcir>_

The dataset is made of collections distinguished by data license at the time of publication:

Each modeling center with bias corrected and downscaled data in this collection falls into one of these license categories - see the table below <#available-institutions-models-and-scenarios-by-license-collection> to see which model is in each collection, and see the section below on Citing, Licensing, and using data produced by this project <#citing-licensing-and-using-data-produced-by-this-project> for citations and additional information about each license. For examples of how to browse the collections and load the data using python, see the example use <#example-use>_ section below.

Data format & contents

The data is stored as partitioned zarr stores (see https://zarr.readthedocs.io <https://zarr.readthedocs.io>_), each of which includes thousands of data and metadata files covering the full time span of the experiment. Historical zarr stores contain just over 50 GB, while SSP zarr stores contain nearly 70GB. Each store is stored as a 32-bit float, with dimensions time (daily datetime), lat (float latitude), and lon (float longitude). The data is chunked at each interval of 365 days and 90 degree interval of latitude and longitude. Therefore, each chunk is (365, 360, 360), with each chunk occupying approximately 180MB in memory.

Historical data is daily, excluding leap days, from Jan 1, 1950 to Dec 31, 2014; SSP data is daily, excluding leap days, from Jan 1, 2015 to either Dec 31, 2099 or Dec 31, 2100, depending on data availability in the source GCM.

The spatial domain covers all 0.25-degree grid cells, indexed by the grid center, with grid edges on the quarter-degree, using a -180 to 180 longitude convention. Thus, the “lon” coordinate extends from -179.875 to 179.875, and the “lat” coordinate extends from -89.875 to 89.875, with intermediate values at each 0.25-degree increment between (e.g. -179.875, -179.625, -179.375, etc).

Available institutions, models, and scenarios by license collection

==================== ================= ========================================== ========================= Modeling institution Source model Available experiments License collection ==================== ================= ========================================== ========================= CAS FGOALS-g3 [] SSP2-4.5, SSP3-7.0, and SSP5-8.5 Public domain datasets INM INM-CM4-8 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 Public domain datasets INM INM-CM5-0 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 Public domain datasets BCC BCC-CSM2-MR SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 CMCC CMCC-CM2-SR5 ssp1-2.6, ssp2-4.5, ssp3-7.0, ssp5-8.5 CC-BY-4.0 CMCC CMCC-ESM2 ssp1-2.6, ssp2-4.5, ssp3-7.0, ssp5-8.5 CC-BY-4.0 CSIRO-ARCCSS ACCESS-CM2 SSP2-4.5 and SSP3-7.0 CC-BY-4.0 CSIRO ACCESS-ESM1-5 SSP1-2.6, SSP2-4.5, and SSP3-7.0 CC-BY-4.0 MIROC MIROC-ES2L SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 MIROC MIROC6 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 MOHC HadGEM3-GC31-LL SSP1-2.6, SSP2-4.5, and SSP5-8.5 CC-BY-4.0 MOHC UKESM1-0-LL SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 MPI-M MPI-ESM1-2-LR SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 MPI-M/DKRZ [] MPI-ESM1-2-HR SSP1-2.6 and SSP5-8.5 CC-BY-4.0 NCC NorESM2-LM SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 NCC NorESM2-MM SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 NOAA-GFDL GFDL-CM4 SSP2-4.5 and SSP5-8.5 CC-BY-4.0 NOAA-GFDL GFDL-ESM4 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0 NUIST NESM3 SSP1-2.6, SSP2-4.5, and SSP5-8.5 CC-BY-4.0 EC-Earth-Consortium EC-Earth3 ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 CC-BY-4.0 EC-Earth-Consortium EC-Earth3-AerChem ssp370 CC-BY-4.0 EC-Earth-Consortium EC-Earth3-CC ssp245 and ssp585 CC-BY-4.0 EC-Earth-Consortium EC-Earth3-Veg ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 CC-BY-4.0 EC-Earth-Consortium EC-Earth3-Veg-LR ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 CC-BY-4.0 CCCma CanESM5 ssp1-2.6, ssp2-4.5, ssp3-7.0, ssp5-8.5 CC-BY-4.0_ ==================== ================= ========================================== =========================

Notes:

.. [*] At the time of running, no ssp1-2.6 precipitation data was available for the FGOALS-g3 model. Therefore, we provide tasmin and tamax for this model and experiment, but not pr. All other model/experiment combinations in the above table include all three variables.

.. [*] The institution which ran MPI-ESM1-2-HR’s historical (CMIP) simulations is MPI-M, while the future (ScenarioMIP) simulations were run by DKRZ. Therefore, the institution component of MPI-ESM1-2-HR filepaths differ between historical and SSP scenarios.

.. _Example Use:

Example Use

See the following examples on github: github.com/microsoft/PlanetaryComputerExamples <https://github.com/microsoft/PlanetaryComputerExamples/blob/main/datasets/cil-gdpcir/>_

You can try these out in a live example on Binder:

.. image:: https://mybinder.org/badge_logo.svg :target: https://mybinder.org/v2/gh/ClimateImpactLab/downscaleCMIP6-binder-env/main?urlpath=git-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252FClimateImpactLab%252FPlanetaryComputerExamples%26urlpath%3Dlab%252Ftree%252FPlanetaryComputerExamples%252Fdatasets%252Fcil-gdpcir%252FREADME.md%26branch%3Dgdpcir-additional-notebooks

.. _Project methods:

Project methods

This project makes use of statistical bias correction and downscaling algorithms, which are specifically designed to accurately represent changes in the extremes. For this reason, we selected Quantile Delta Mapping (QDM), following the method introduced by Cannon et al. (2015) <https://doi.org/10.1175/JCLI-D-14-00754.1>_, which preserves quantile-specific trends from the GCM while fitting the full distribution for a given day-of-year to a reference dataset (ERA5).

We then introduce a similar method tailored to increase spatial resolution while preserving extreme behavior, Quantile-Preserving Localized-Analog Downscaling (QPLAD).

Together, these methods provide a robust means to handle both the central and tail behavior seen in climate model output, while aligning the full distribution to a state-of-the-art reanalysis dataset and providing the spatial granularity needed to study surface impacts.

A publication providing additional detail is in review for publication in Geoscientific Model Development and a pre-print can be accessed in EGUsphere: https://egusphere.copernicus.org/preprints/2023/egusphere-2022-1513/.

.. _The downscaleCMIP6 Repository:

The downscaleCMIP6 Repository

The ClimateImpactLab/downscaleCMIP6 <https://github.com/ClimateImpactLab/downscaleCMIP6>_ repository contains infrastructure setup, argo workflows, and validation notebooks which together produce the bias corrected and downscaled daily 1/4-degree CMIP6 tasmin, tasmax, and pr data for the Climate Impact Lab Global Downscaled Projections for Climate Impacts Research (CIL GDPCIR) project.

See also:

.. _Citing, licensing, and using data produced by this project:

Citing, licensing, and using data produced by this project

Projects making use of the data produced as part of the Climate Impact Lab Global Downscaled Projections for Climate Impacts Research (CIL GDPCIR) project are requested to cite both this project and the source datasets from which these results are derived. Additionally, the use of data derived from some GCMs requires citations. See each GCM's license info in the links below for more information.

.. _CIL GDPCIR:

CIL GDPCIR

Users are requested to cite this project in derived works.

Gergel, D. R., Malevich, S. B., McCusker, K. E., Tenezakis, E., Delgado, M. T., Fish, M. A., and Kopp, R. E.: Global Downscaled Projections for Climate Impacts Research (GDPCIR): preserving quantile trends for modeling future climate impacts, Geosci. Model Dev., 17, 191–227, https://doi.org/10.5194/gmd-17-191-2024, 2024.

.. _ERA5:

ERA5

Additionally, we request you cite the historical dataset used in bias correction and downscaling, ERA5. See the ECMWF guide to citing a dataset on the Climate Data Store <https://confluence.ecmwf.int/display/CKB/How+to+acknowledge+and+cite+a+Climate+Data+Store+%28CDS%29+catalogue+entry+and+the+data+published+as+part+of+it>_:

Hersbach, H, et al. The ERA5 global reanalysis. Q J R Meteorol Soc.2020; 146: 1999–2049. https://doi.org/10.1002/qj.3803

Muñoz Sabater, J., (2019): ERA5-Land hourly data from 1981 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS). (Accessed on June 4, 2021), 10.24381/cds.e2161bac

Muñoz Sabater, J., (2021): ERA5-Land hourly data from 1950 to 1980. Copernicus Climate Change Service (C3S) Climate Data Store (CDS). (Accessed on June 4, 2021), 10.24381/cds.e2161bac

.. _GCM-specific citations & licenses:

GCM-specific citations & licenses

The CMIP6 simulation data made available through the Earth System Grid Federation (ESGF) are subject to Creative Commons BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0/> or BY-NC-SA 4.0 <https://creativecommons.org/licenses/by-nc-sa/4.0/> licenses. We have reached out to each of the modeling institutions to request waivers from these terms so the outputs of this project may be used with fewer restrictions, and have been granted permission to release our data using the licenses listed here.

.. _CC0:

Public Domain Datasets


The following bias corrected and downscaled model simulations are available in the public domain using a `CC0 1.0 Universal Public Domain Declaration <https://creativecommons.org/publicdomain/zero/1.0/>`_. Access the collection on Planetary Computer at https://planetarycomputer.microsoft.com/dataset/cil-gdpcir-cc0.

* **FGOALS-g3**

  License description: `data_licenses/FGOALS-g3.txt <https://raw.githubusercontent.com/ClimateImpactLab/downscaleCMIP6/master/data_licenses/FGOALS-g3.txt>`_

  CMIP Citation:

    Li, Lijuan **(2019)**. *CAS FGOALS-g3 model output prepared for CMIP6 CMIP*. Version 20190826. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1783

  ScenarioMIP Citation:

    Li, Lijuan **(2019)**. *CAS FGOALS-g3 model output prepared for CMIP6 ScenarioMIP*. SSP1-2.6 version 20190818; SSP2-4.5 version 20190818; SSP3-7.0 version 20190820; SSP5-8.5 tasmax version 20190819; SSP5-8.5 tasmin version 20190819; SSP5-8.5 pr version 20190818. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2056

* **INM-CM4-8**

  License description: `data_licenses/INM-CM4-8.txt <https://raw.githubusercontent.com/ClimateImpactLab/downscaleCMIP6/master/data_licenses/INM-CM4-8.txt>`_

  CMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana **(2019)**. *INM INM-CM4-8 model output prepared for CMIP6 CMIP*. Version 20190530. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1422

  ScenarioMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana **(2019)**. *INM INM-CM4-8 model output prepared for CMIP6 ScenarioMIP*. Version 20190603. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.12321

* **INM-CM5-0**

  License description: `data_licenses/INM-CM5-0.txt <https://raw.githubusercontent.com/ClimateImpactLab/downscaleCMIP6/master/data_licenses/INM-CM5-0.txt>`_

  CMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana **(2019)**. *INM INM-CM5-0 model output prepared for CMIP6 CMIP*. Version 20190610. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1423

  ScenarioMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana **(2019)**. *INM INM-CM5-0 model output prepared for CMIP6 ScenarioMIP*. SSP1-2.6 version 20190619; SSP2-4.5 version 20190619; SSP3-7.0 version 20190618; SSP5-8.5 version 20190724. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.12322

.. _CC-BY:

CC-BY-4.0

The following bias corrected and downscaled model simulations are licensed under a Creative Commons Attribution 4.0 International License <https://creativecommons.org/licenses/by/4.0/>_. Note that this license requires citation of the source model output (included here). Please see https://creativecommons.org/licenses/by/4.0/ for more information. Access the collection on Planetary Computer at https://planetarycomputer.microsoft.com/dataset/cil-gdpcir-cc-by.

Acknowledgements

This work is the result of many years worth of work by members of the Climate Impact Lab <https://impactlab.org>_, but would not have been possible without many contributions from across the wider scientific and computing communities.

Specifically, we would like to acknowledge the World Climate Research Programme's Working Group on Coupled Modeling, which is responsible for CMIP, and we would like to thank the climate modeling groups for producing and making their model output available. We would particularly like to thank the modeling institutions whose results are included as an input to this repository (listed above) for their contributions to the CMIP6 project and for responding to and granting our requests for license waivers.

We would also like to thank Lamont-Doherty Earth Observatory, the Pangeo Consortium <https://github.com/pangeo-data> (and especially the ESGF Cloud Data Working Group <https://pangeo-data.github.io/pangeo-cmip6-cloud/#>) and Google Cloud and the Google Public Datasets program for making the CMIP6 Google Cloud collection <https://console.cloud.google.com/marketplace/details/noaa-public/cmip6> possible. In particular we're extremely grateful to Ryan Abernathey <https://github.com/rabernat>, Naomi Henderson <https://github.com/naomi-henderson>, Charles Blackmon-Luca <https://github.com/charlesbluca>, Aparna Radhakrishnan <https://github.com/aradhakrishnanGFDL>, Julius Busecke <https://github.com/jbusecke>, and Charles Stern <https://github.com/cisaacstern>_ for the huge amount of work they've done to translate the ESGF CMIP6 netCDF archives into consistently-formattted, analysis-ready zarr stores on Google Cloud.

We're also grateful to the xclim developers <https://github.com/Ouranosinc/xclim/graphs/contributors> (DOI: 10.5281/zenodo.2795043 <https://doi.org/10.5281/zenodo.2795043>), in particular Pascal Bourgault <https://github.com/aulemahal>, David Huard <https://github.com/huard>, and Travis Logan <https://github.com/tlogan2000>, for implementing the QDM bias correction method in the xclim python package, supporting our QPLAD implementation into the package, and ongoing support in integrating dask into downscaling workflows. For method advice and useful conversations, we would like to thank Keith Dixon, Dennis Adams-Smith, and Joe Hamman <https://github.com/jhamman>.

Financial support

This research has been supported by The Rockefeller Foundation and the Microsoft AI for Earth Initiative.