julietcohen
commented
10 months ago A version of the data linked above already exists on Datateam, where it is used for the "how weird is the weather" project. However, the data on Datateam is not cloud optimized. Anna and Matt clarified that this data is not intended to be fed into the viz workflow for mapping and visualization as a normal layer. Instead, it will be used for user interface tools on the PDG, such as the plot viewer. When a user selects a certain region of the map in the portal, the many data points for the time series climate for that region can be quickly queried on the backend and fed to metacatUI.
ERA5 climate dataset
Historical climate variables spanning from 1940 to 2023 are available through the fifth generation of the European Centre for Medium-Range Weather Forecasts' Atmospheric Reanalysis, according to the Google Cloud Public Datasets documentation. These reanalyses were created from assimilation of various sources via numerical weather prediction models. This dataset includes hourly estimates for atmospheric, land, and oceanic climate variables at 30m resolution and is several petabytes in size. Google has produced a cloud-optimized version of ERA5 data specifically for the variables related to the atmosphere and land. This data is in Zarr format. The Google Cloud analysis-ready corpus covers the years 1959-2022.
This time series dataset would be interesting to combine with other time series datasets on the PDG, such as annual lake area. Because Zarr data is large N-dimensional arrays and not vector data, this dataset is not suitable for input into the visualization workflow at the staging step this time.
GitHub repository
xarrayto access the Zarr data in an example notebook.Google Cloud Public Datasets
Data use license
Citation: Carver, Robert W, and Merose, Alex. (2023): ARCO-ERA5: An Analysis-Ready Cloud-Optimized Reanalysis Dataset. 22nd Conf. on AI for Env. Science, Denver, CO, Amer. Meteo. Soc, 4A.1, https://ams.confex.com/ams/103ANNUAL/meetingapp.cgi/Paper/415842