The code in this repository provides a scalable and flexible framework to apply convolutions on spherical unstructured grids for weather/climate applications.
ATTENTION: The code is subject to changes in the coming weeks / months.
The folder experiments
(will) provide examples for:
The folder tutorials
(will) provide jupyter notebooks describing various features of DeepSphere-Weather.
The folder docs
(will) contains slides and notebooks explaining the DeepSphere-Weather concept.
For a local installation, follow the below instructions.
Clone this repository.
git clone https://github.com/deepsphere/deepsphere-weather.git
cd deepSphere-weather
Install manually the following dependencies:
conda install -c conda-forge pytorch-gpu
conda install -c conda-forge pytorch-cpu
conda create --name weather python=3.8
conda install xarray dask cdo h5py h5netcdf netcdf4 zarr numcodecs rechunker xskillscore
conda install notebook jupyterlab
conda install matplotlib-base cartopy pycairo seaborn cycler
conda install numpy pandas numba scipy bottleneck
conda install yaml tabulate tqdm deepdiff
conda install healpy igl shapely
pip install git+https://github.com/epfl-lts2/pygsp@sphere-graphs
pip install torchinfo
git clone git@github.com:ghiggi/xverif.git
git clone git@github.com:ghiggi/xscaler.git
git clone git@github.com:ghiggi/xsphere.git
git clone git@github.com:ghiggi/xforecasting.git
Alternatively install the dependencies using one of the appropriate below environment.yml files:
conda env create -f environment_python3.8.5.yml
conda env create -f environment_python3.9.yml
and clone the required repository:
git clone git@github.com:ghiggi/xverif.git
git clone git@github.com:ghiggi/xscaler.git
git clone git@github.com:ghiggi/xsphere.git
git clone git@github.com:ghiggi/xforecasting.git
Add the PYTHONPATH (i.e. in the .bashrc) for the following packages. Here is an example:
export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xscaler"
export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xverif"
export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xsphere"
export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xforecasting"
export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Projects/deepsphere-weather"
spherical_grids.ipynb
: get a taste of samplings/pixelizations of the sphere.interpolation_pooling.ipynb
: get to understand our generalized pooling based on interpolation between samplings.Here we are going to document the data structure
The data required for model training and evaluation are stored with the following folder hierarchy.
preprocessed
└───ERA5_HRES
| └─── <sampling>
| └─── Data
| └─── Scalers
| └─── Benchmarks
| └─── Climatology
|
└───IF5_HRES
| └─── <sampling>
| └─── Data
|
└───IF5_ENS
| └─── <sampling>
└─── Data
On the LTE servers, data are available in the directory /ltenas3/data/DeepSphere
The data are available upon request to the authors.
The content of this repository is released under the terms of the MIT license.