Open tinased95 opened 1 year ago
Hi @tinased95,
Regarding missing data:
Some solutions:
Let us know.
Thank you so much for your helpful solutions. Could you please elaborate more on your second solution? How can I use the multiple unconnected series for the model?
The idea for unconnected time series, would operate as follows: Suppose that you have a series $y[:t]=[y{1},y{2},...,y{t}]$, and it has two connected blocks, with missing values in between $y[:t1]=[y{1},y{2},...,y{t1}]$ and $y[t2:t]=[y{1},y{2},...,y_{t1}]$ you could hack the problem and treat these blocks as individual series in the panel. Only keeping the predictions for $y[t2:t]$. The idea might be a stretch of NeuralForecast functionalities, I would try first filtering or dig into SpectraNet.
Thanks, I'm not sure how to pass these individual series into the model's fit() or predict() functions. Should they be treated as different targets i.e. multi-variate forecasting? but in that case the 'ds' which is for the time would not be aligned between the series.
Hi @tinased95. We added the possibility of specifying an available_mask
in the input dataframe (as a new column, 1:available, 0:missing). You can impute missing values (so models can use them as inputs). The mask will weight the training loss to prevent learning imputed values.
Hi @cchallu, that's great. Thank you so much for letting me know.
Hi, I was also wondering how missing rows were handled in windows-sampling models.
I thought that the idea was that each sample would contain n_past + n_horizon
rows, where the contribution of values (past and future) from rows where available_mask==0
would be ignored towards the loss computation. Looking at how _compute_weights()
is used by the losses, it looks like this is implemented simply by zeroing out the contributing target(s) with a weight of 0.
Thanks
Hello, my time series data has some large missing gaps, I was wondering if it is possible to remove the rows with missing value instead of applying data interpolation methods to clean the data? Should the data be continuous or is it okay to not have some rows in the data? Thanks.