search

ServiceNow / TACTiS

TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series, from ServiceNow Research
Apache License 2.0
110 stars 19 forks source link

Multi input example? #17

Open kevindarby opened 1 year ago

kevindarby commented 1 year ago

Hi,

Do you have an example by which the model predicts the future value of one input from a dataset containing this input and many other features?

I saw in the paper you tested on kdd-cup, but I only see single input series in your demo/ folder

If you don't have such an example, could you please recommend some changes I can make to try?

It seems that hist_value and pred_value must be of the same dimension, and that's where I'm having trouble

Excellent paper, thank you.

marcotet commented 1 year ago

Hi,

We sadly do not have an example where we add covariates to the data when doing the forecasting. We haven't tested that functionality, although theoretically the model should be compatible with it.

If you want to change the model, you should start with modifying the code which combine the values, the series embedding, and the mask (1 or 0) values. These create the hist_encoded and pred_encoded variables: https://github.com/ServiceNow/tactis/blob/1ddcde44730ce0c0188e66e060fcb643cf70028e/tactis/model/tactis.py#L410 for the loss, and https://github.com/ServiceNow/tactis/blob/1ddcde44730ce0c0188e66e060fcb643cf70028e/tactis/model/tactis.py#L512 for the sampling.

Note that these encodings are sent to a MLP before going to the Transformer layers, so if you adjust their dimensionality, you will have to adjust the first layer of said MLP too: https://github.com/ServiceNow/tactis/blob/1ddcde44730ce0c0188e66e060fcb643cf70028e/tactis/model/tactis.py#L296

Good luck with your experiments, and thank you for your interest in our model.

kevindarby commented 1 year ago

Thanks, after a re-read of your paper, I wonder if I could obtain results similar to using covariates by treating each covariate as its own time series. Did you do this with kdd-cup or only train on targets for the paper?

marcotet commented 1 year ago

It would indeed be a possible method to implement covariates. However, it would require some refactoring, since you would have to make sure that the mask for these covariates is correctly set, and their values are not set to 0 in the forecasting window.

But we haven't tried to do this so far. I would expect that doing so would require a bigger model than concatenating the covariates to the initial input, since the only way these covariates could interact with the variables to be forecasted will be through the attention mechanism, which does have limited capacity for smaller model sizes.