Open leifdenby opened 2 weeks ago
Methods:
from_file(filepath)
, called in:
calculate_statistics.main
with filepath=args.data_config
create_boundary_mask.main
with filepath=args.data_config
create_forcings.main
with filepath=args.data_config
create_mesh.main
with filepath=args.data_config
neural_lam.models.ar_model.ARModel.__init__
with filepath=args.data_config
neural_lam.weather_dataset.WeatherDataset.__init__
with filepath=args.data_config
plot_graph.main
with filepath=args.data_config
process_dataset(category, split, apply_windowing=True)
, called in
neural_lam.models.ar_model.ARModel.__init__
with category="static"
neural_lam.models.weather_dataset.WeatherDataset
with category="state"
and category="forcing"
open_zarrs(category)
, called in:
create_forcings.main
with category="state"
neural_lam.config.Config
class itselfget_xy(category, stacked=True)
, called in:
create_mesh.main
with category="static"
neural_lam.config.Config.get_xy_extent
it self with stacked=False
plot_graph
with category="state"
load_normalization_stats(category, datatype="torch")
, called in:
neural_lam.models.ARModel.__init__
with category="state", datatype="torch"
neural_lam.weather_dataset.WeatherDataset.__init__
with category="state", datatype="torch"
and category="forcing", datatype="torch"
load_boundary_mask()
, called in:
neural_lam.config.Config.__init__
num_data_vars(category)
, called in:
neural_lam.models.ARModel.__init__
with category="state"
three times and with category="forcing"
oncevars_names(category)
called in
neural_lam.config.Config._select_stats_by_category(combined_stats, category)
twiceneural_lam.models.ar_model.ARModel.plot_examples
with category="state"
twiceneural_lam.models.ar_model.ARModel.create_metric_log_dict
with category="state"
vars_units(category)
, called in:
neural_lam.models.ar_model.ARModel.plot_examples
with category="state"
Attributes:
grid_shape_state
, defined in neural_lam/data_config.yml
as grid_shape_state=dict(y=589, y=789)
, referenced in
neural_lam.vis.plot_prediction
, neural_lam.vis.plot_spatial_error
and create_boundary_mask.main
forcing
accesses entire sub-tree dict structure forcing
in neural_lam/data_config.yml
, referenced in
neural_lam.models.ar_model.ARModel.__init__
step_length
computed time coordinate values of state
"category", referenced in
neural_lam.models.ar_model.ARModel.__init__
coords_projection
, defined in root of neural_lam/data_config.yml
, referenced in
neural_lam.vis.plot_prediction
and neural_lam.vis.plot_spatial_error
@sadamov I made a dataclasses based class structure which matches the config yaml file you've made: https://github.com/leifdenby/neural-lam/blob/mllam-dataloader/neural_lam/multizarr_datastore_config.py
I will explain this in more detail tomorrow, but the idea is that by using dataclasses the config structure is stored in code, it is validated, and the nested structure (enabling the dot-syntax nesting that you achieve with __getattr__
) is provided through the nested dataclass objects. It doesn't match everything because I don't quite understand the whole config. The serialization to/from yaml is handled by dataclass-wizard
I am in the process of reading through @sadamov's PR #54 on using zarr-based datasets in
neural-lam
and I am going to use this issue to write down some notes. Everyone is free to read-along, but this will only over time become a coherent piece of information, so it is probably best to wait until I comment directly on #54.Uses of current
neural_lam.config.Config
attributes and methods outside the class itself: