sadamov
commented
1 month ago Suggestion how the projection and regional extent can be specified:
projections:
lambert_conformal_conic:
proj_class: LambertConformal
proj_params:
central_longitude: 15.0
central_latitude: 63.3
standard_parallels: [63.3, 63.3]
mercator:
proj_class: Mercator
proj_params:
central_longitude: 0.0
min_latitude: -80.0
max_latitude: 84.0
stereographic:
proj_class: Stereographic
proj_params:
central_longitude: 0.0
central_latitude: 90.0
true_scale_latitude: 60.0
rotated_pole:
proj_class: RotatedPole
proj_params:
pole_longitude: 10.0
pole_latitude: -43.0
robinson:
proj_class: Robinson
proj_params:
central_longitude: 0.0
plate_carree:
proj_class: PlateCarree
proj_params:
central_longitude: 0.0
limits:
x_limits: [-6.82, 4.8]
y_limits: [-4.42, 3.36]
ThomasRieutord
joeloskarsson
TomasLandelius
leifdenby
This supersedes #2 and #3.
Motivation
It is currently very hard to work with neural lam on different regions due to everything related to data and the forecast region being specified hard-coded in
constants.py. It would be much better to specify this in a config file that you can then point to. Yaml seems like a suitable format for this.Proposition
The main training/eval script takes a flag
--specthat should be given a path to a yaml-file. This yaml file specifies all the data that goes into the model and information about the region the model should be working with.Current options in
constants.pythat relate to what to plot should all be turned into flags.The yaml file should be read in and turned into a single object that contains all useful information and can be passed around in the code (since this is needed almost everywhere). Having this as an object means that it can also compute things not directly in the yaml file, such as units of variables that can be retrieved from loaded xarray data.
Design
Here is an idea for how the yaml file could be laid out with examples:
Start of file:
Some comments to keep track of what this specification is about. We don't enforce any content in there.
Forecast area data configuration
This describes the data configuration for the actual limited area that you are training on. Explicitly, the "inner region", not the "boundary". What is specified is what zarrs to load state, forcing and static grid features from and which variables in these to use for each.
griddimension or the name ofxandydimensions. Ifxandythen we will flatten ourselves in the data loading code. If onlygridis specified, agrid_shapeentry has to be given (see below).Boundary data configuration
The boundary data configuration follows exactly the same structure as the forecast_area, with two differences:
stateentry is allowed, as we do not forecast the boundary nodes atm.Grid shape
If the zarrs already contain flattened grid dimensions we need knowledge of the original 2d spatial shape in order to be able to plot data. For such cases this can be given by an optional
grid_shapeentry:Subset splits
The train/val/test split is defined based on timestamps:
Used by the dataset class to
.selthe different subsets.Forecast area projection
In order to be able to plot data in the forecasting area we need to know what projection the area is defined in. By plotting in this projection we end up with a flat rectangular area where the data sits. This should be specified as a reference to a cartopy.crs object.
Normalization zarr
We also need information about statistics of variables, boundary and forcing for normalization (mean and std). Additionally we need the inverse variances used in the loss computation. As we compute and save this in a pre-processing script we can enforce a specific format, so lets put all of those also in its own zarr. Then we only need to specify a path here to that zarr to load it from.