search

BigDataWUR / AgML-CY-Bench

CY-Bench (Crop Yield Benchmark) is a comprehensive dataset and benchmark to forecast crop yields at subnational level. CY-Bench standardizes selection, processing and spatio-temporal harmonization of public subnational yield statistics with relevant predictors. Contributors include agronomers, climate scientists and machine learning researchers.
https://cybench.agml.org/
Other
18 stars 8 forks source link

nn models rewrite #289

Closed krsnapaudel closed 3 months ago

krsnapaudel commented 3 months ago

Tried to simplify nn-models code.

  1. Skip early stopping and saving best model.
  2. Hyperparameter optimization and confusion about what is in settings, args and kwargs.
  3. predict_batch and predict.
  4. Transforms code. Separating time series and static inputs must be done even when transforms is empty.
  5. Add validation of AgML Workshop 2024 results.
krsnapaudel commented 3 months ago

@aikepotze: "However, this would need changes to the class structure, as the base nn model cannot re-initialize itself. What do you think?" I don't think the model needs to reinitialize itself. We have saved init_args for models we create to optimize hyperparameters. That I think is sufficient.

krsnapaudel commented 3 months ago

I have validated results from AgML workshop (#261). Now the workshop implementation of LSTMModel and ExampleLSTM included in the benchmark have similar results when using workshop data. The architectures are similar. Some differences:

  1. LSTMModel relies on batch norm for normalizing inputs and intermediate features. ExampleLSTM normalizes inputs on its own. There is no batch norm after LSTM layer.
  2. LSTMModel does not use LRScheduler; ExampleLSTM does.

The NRMSE on test years [2012, 2018] is approximately 25% for both models.