Closed diegoquintanav closed 3 years ago
Hi @diegoquintanav,
I've built a quick gist with a dummy dataset to demonstrate how both single-step and multi-step forecasting problems could be implemented with tsai
.
As you'll see the key is to build the target with the expected shape. tsai
will recognize the shape of the target and will create an output of the same shape.
Questions:
I'm not sure what you mean when you say "but it breaks other parts of the API.". Could you please elaborate on that?
I'm in the process of creating 1 or 2 more detailed examples to demonstrate how to use forecasting using tsai
.
Hi @oguiza, and thanks for the reply!
idx=1
argument, It means then that the forecast starts from the index=1 in the splits
array, which is the validation set. In other words, it is equivalent to pass X[splits[1]]
to get_X_preds
, as shown in the notebook. Let me know if my understanding is correct. The main difference I see about multistep forecasting is the value for horizon
> 1. How does one then interpret the plots in the last cell?
for example, in
>>> valid_decoded_preds[0]
tensor([0.4506, 0.4616, 0.2979])
valid_decoded_preds[1]
contains the same but for t=1?valid_preds, valid_targets_inc, valid_decoded_preds
in the output of get_X_preds
? More specifically, what is the difference between decoded and not decoded preds?About the API breaking, I can't reproduce the issue right now, but some methods implemented in the learner were not working if I used TSForecasting
instead of TSRegression
. Don't mind that for now. I will open another issue if I can reproduce the problem again.
Thanks again for your time and for the library too!
edit: I understood the meaning of splits[1]
So I think an autoregressive forecast would look something like this
# name aliasing for local referencing
seq_in_len = window_length
seq_out_len = horizon
# empty placeholder
preds = np.repeat(np.nan, len(splits[1]) + seq_in_len + seq_out_len - 1)
# seed first values
preds[:seq_in_len] = X[splits[1]][0].flatten()
for ix in range(len(splits[1])):
new_x = preds[: ix + seq_in_len] # alternatively
_, _, _valid_decoded_preds_inc = learn_inc.get_X_preds(new_x)
# get size of last output
_h = _valid_decoded_preds_inc.flatten().shape[0]
# replace values in placeholder array
preds[ix + seq_in_len : ix + seq_in_len + _h] = _valid_decoded_preds_inc.flatten()
If I do this with the InceptionTime
model, I get something like
fig, ax = plt.subplots(figsize=(15, 7))
ax.plot(preds, label="decoded_preds (AR)")
ax.plot(X[splits[1]][0, 0, :].tolist(), marker="o", label="first window")
ax.plot([np.nan]*window_length + valid_decoded_preds_inc[:, 0].tolist(), label="decoded_preds (Window ahead)")
ax.plot([np.nan]*window_length + y[splits[1]][:, 0].tolist(), label="targets")
fig.legend()
Which looks more like an autoregressive forecast (values are not relevant). Tell me what you think :+1:!
Hola Diego,
I believe it's correct, but I don't fully understand everything in your code. For example, with this code:
for ix in range(len(splits[1])):
new_x = preds[: ix + seq_in_len]
new_x is increasingly larger. That doesn't make much sense to me if you have trained the model with equally long inputs.
Having said that, in my experience, results are better with a multiple output forecast (that is, creating the forecast for the entire horizon simultaneously). It'd also be easier to create in tsai
. You just need to pass a target with the desired horizon length.
Right, considering I'm doing a forecast at time t[i] = t_i
, I will produce t[i+1:t+horizon]
. The model was trained using t[i-window_length:i], so a better way would be something like
new_x = preds[ix: ix + seq_in_len]
that fixes the input dimensionality to window_length
. I'm not sure which one is better though. It is true that it is not the way the model was trained, and it produces a totally different output that does not damp over time.
About the multiple output forecast (or multi-step forecast), I believe that the case I'm proposing is the recursive multistep forecast (2)
prediction(t+1) = model(obs(t-1), obs(t-2), ..., obs(t-n))
prediction(t+2) = model(prediction(t+1), obs(t-1), ..., obs(t-n))
and what you suggest is number (4), by setting a large enough number in the horizon
argument. In this case, I have many questions about the model itself (I have had trouble understanding the underlying lightning API for training :sweat_smile: )
TransformerModel
, does this mean that the encoder is fed window_length
data and the decoder is fed horizon
during training? What happens during inference?valid_decoded_preds
and what does decoded mean?I'll try to answer your questions. But before I have a few comments:
As to your questions:
TransformerModel
doesn't have a decoder. It only has an encoder. You only need to ensure X (input) and y (output) have the desired shape. X: [n_samples x n_variables x history] and y: [n_samples x horizon] for univariate and [n_samples x n_variables x horizon] for multivariate. tsai
will automatically create a head that will generate the expected output shape.
This applies to all models, not just to TransformerModel
. You can also, for example, try this approach with InceptionTime
or TST
. Hey, thanks for answering! Everything is more clear now. I will close the issue.
Hi! from #125 it's not yet clear to me how a forecasting problem looks like. I've noticed there is a
TSForecasting
class at the implementation level, which is the same asTSRegression
(both are equally set toToFloat
) but it breaks other parts of the API.See https://github.com/timeseriesAI/tsai/blob/aa8b32a50d52692355214c35cb140f586600db66/tsai/data/core.py#L112
How does inference work in a forecasting example? From the examples I use
learner.get_preds(ds_idx=1)
but how is this working internally? It usesfastai.learner.Learner.get_preds
which I have trouble following :sweat_smile:.In other words, consider the following gist
And the output of
check_data(X, y, splits)
Questions:
Thanks!