mononitogoswami
commented
4 months ago Hi Satyajit, Thanks for your interest in MOMENT!
MOMENT can be used for time series shorter than 512 time steps! In fact, most time series in the Time Series Pile are shorter than 512 time steps!
Here’s one way to model shorter time series:
MOMENT accepts time series and a boolean input_mask as input. The input mask is a boolean mask of shape (512,). In this mask, 1 indicates time steps that MOMENT must attend to, and 0 indicates time steps that MOMENT should ignore.
Let's say we have a short time series of length 64. You can left pad this time series with 448 zeros, and pass it along with a input_mask of length with 448 0s followed by 64 1s.
Finer details
MOMENT learns representations for patches of time series data rather than modeling each time step individually. Patches are disjoint time series sub-sequences of fixed length (patch_len = 8) . A time series of length 512, therefore has 64 patches of length 8. The representations of all these patches are mapped to the forecasting horizon using a linear forecasting head (64 x 1024 => forecasting_horizon). This linear forecasting head is randomly initialized and must be fine-tuned on some data. For a shorter time series of length 64, one can only take the representations corresponding to 8 patches and use a smaller forecasting head (8 x 1024 => forecasting_horizon).
It is important to note that MOMENT masks an entire patch if even one time step is masked. So for a time series of length which is not a multiple of 8 (say 61), the left-most patch may be masked. To prevent loss of information, it may be useful to left-pad the values on this patch (to extend the length of the time series to 64) by simply repeating the last observation until the patch boundary.
Let us know if you have more questions, and thanks again for your interest in MOMENT! If this answers your question, feel free to close the issue :-)
Satyajit-Chaudhuri
Hello Developers, Congrats for your wonderful work.
Now in the Forecasting tutorial notebook it is mentioned that "Moment takes an input time series of length T=512." Now it is very difficult to obtain such a large dataset for most industrial use cases where monthly and weekly forecasts are required. How to handle that?