Setup
All instructions are from the root of the repository, i.e. the folder containing this README.md file.
This code has only been tested on macos and linux. Equivalent commands may work on windows, but I cannot guarantee it.
Pre-requisites
You will need to install conda as some of the packages are harder to install with pip.
Create A virtual environment from the environment.yml file
conda env create -f environment.ymlActivate the virtual environment
conda activate sim2realInstall the sim2real package locally
This will allow you to import the sim2real package from anywhere on your system, and allows you to run files within the sim2real package via IPython or Jupyter.
pip install -e .Install the DWD data
Note, this downloads the current data from the DWD website, which has changed in the past. This might lead to unexpected downstream behaviour as the set of stations, dates, and the data itself changes. If you want to use the data from the paper, please contact the authors.
python sim2real/preprocessing/dwd.pyDownload the ERA5 data
This is not automated yet, but you can download the relevant data from the ERA5 website here.
The following screenshots show the settings you should use to download the data:
Alternatively, if you have the cdsapi package installed and configured, you can use the following python code to download the data:
Click to Expand
```python import cdsapi c = cdsapi.Client() c.retrieve( 'reanalysis-era5-single-levels', { 'product_type': 'reanalysis', 'variable': '2m_temperature', 'year': [ '2012', '2013', '2014', '2015', '2016', '2017', '2018', '2019', '2020', '2021', '2022', ], 'month': [ '01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12', ], 'day': [ '01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', ], 'time': [ '00:00', '01:00', '02:00', '03:00', '04:00', '05:00', '06:00', '07:00', '08:00', '09:00', '10:00', '11:00', '12:00', '13:00', '14:00', '15:00', '16:00', '17:00', '18:00', '19:00', '20:00', '21:00', '22:00', '23:00', ], 'area': [ 54.95, 5.8, 47.2, 15.05, ], 'format': 'grib', }, '1.grib') ``` This should download a file called `1.grib` to your current working directory.You should then move the downloaded 1.grib file to the path specified in sim2real/config.py under the raw_era5 path. By default, this is the folder created by
mkdir -p data/raw/ERA_5_GermanyThen move 1.grib to data/raw/ERA_5_Germany/1.grib, or adjust raw_era5 in sim2real/config.py to point to the correct location.
Finally, process the file by running
python sim2real/preprocessing/era5.pyDownload the elevation data
This is also not automated yet. Please go to the opendem SRTM data download and download the SRTM "Digital Terrain Model of Germany".
This should download a file called srtm_germany_dtm.zip, which you should unzip into srtm_germany_dtm.tif. This tif file should be moved to the path specified in sim2real/config.py under the raw_srtm key. By default this is
mkdir -p data/raw/SRTM_DEMFinally, process the file by running
python sim2real/preprocessing/srtm_dem.py