search

google-research / tuning_playbook

A playbook for systematically maximizing the performance of deep learning models.
Other
26.29k stars 2.18k forks source link

Question: Time Series Missing Data? #46

Closed fdelcab closed 5 months ago

fdelcab commented 1 year ago

Hi everyone,

I would like to know your opinion about seqtopoint deep learning models for forecasting and missing data. I am dealing with a time series that has a lot of time discontinuity data, so when I resample the data, e.g. to 1 minute, I end up with a bunch of missing gaps.

  1. Should I include them in the input data (e.g. for a LSTM model) so I don't loose the time continuity characteristic of my time series?
  2. Will the NaN values will help in the generalization of the model, since they can be related with noisy data?
  3. How does LSTM model handles missing data (NaN values)?

Thank you for your time

joker507 commented 1 year ago

Missing values can be completed using a variety of strategies, such as 0 fill, linear fill, fill with adjacent values, etc. There may be more advanced methods. I don't know how to evaluate performance between these methods

varungodbole commented 5 months ago

Hey, this is a really great question. We believe that it's a bit outside the scope of the playbook, and we don't have any specific recommendations. So we're closing this issue for now.