search

hydrogo / rainnet

RainNet: a convolutional neural network for radar-based precipitation nowcasting
MIT License
120 stars 61 forks source link

no check for consecutive times lots when training #11

Closed JKupzig closed 2 years ago

JKupzig commented 2 years ago

I've noticed that the get_item method in the dataset class does not proof if the 5 np.arrays used for training are actually 5 consecutive timeslots. I'm referring to this line:

arr = np.array(self.dataset.get(self.keys[i+j])) and y = np.squeeze(processor.data_preprocessing(np.array(self.dataset[self.keys[i+4]])[np.newaxis,:,:])) #corrected after github issue

E.g. the first 3 arrays can be the 01.01.2020 5:00, 01.01.2020 5:05, 01.01.2020 5:10 but the next is 01.01.2020 5:20 and should not be used.

hydrogo commented 2 years ago

Hi @JKupzig, I think one should check that before the training while constructing a dataset (like in the notebook provided by @PradyumnaGupta). I did that by creating a dictionary with valid input-output timestamp pairs that then I fed to the Dataset class.

JKupzig commented 2 years ago

Thank you for the brief explanation @hydrogo!