search

BigDataWUR / AgML-CY-Bench

CY-Bench (Crop Yield Benchmark) is a comprehensive dataset and benchmark to forecast crop yields at subnational level. CY-Bench standardizes selection, processing and spatio-temporal harmonization of public subnational yield statistics with relevant predictors. Contributors include agronomers, climate scientists and machine learning researchers.
https://cybench.agml.org/
Other
18 stars 8 forks source link

save aligned data for faster data loading #257

Open krsnapaudel opened 5 months ago

krsnapaudel commented 5 months ago
  1. Align with crop calendar
  2. Align indices of labels and predictors I realized that aligning with lead time must be done afterwards. So the current code in trim_to_lead_time must be split into two parts: do 1 and 2 above and save data. Do the trimming to lead time later.
krsnapaudel commented 23 hours ago

The code to save and load aligned data was removed in a subsequent merge. Reopening.

Considerations:

  1. Data path vs aligned data. Is the aligned data path inside data path?
  2. Should we reconsider how data is organized now, i.e. crop and then country?
  3. We should also save data after TorchDataset interpolates and aggregates time series. Should we make it a separate issue?