search

awslabs / gluonts

Probabilistic time series modeling in Python
https://ts.gluon.ai
Apache License 2.0
4.57k stars 750 forks source link

Specifying Loss Distributions for Temporal Fusion Transform #2168

Open esbraun opened 2 years ago

esbraun commented 2 years ago

Description

Currently the mxnet temporal fusion transformer only allows for quantile loss. It would be helpful to be able to specify other distributions (i.e. negative binomial for count data).

Would this change be possible?

References

The library pytorch_forecasting allows specification of the loss for its temporal fusion transformer implementation.

kashif commented 2 years ago

note I have such a version for pytorch: https://github.com/kashif/pytorch-transformer-ts/tree/main/tft

esbraun commented 2 years ago

Thank you! Is this something that you're planning on merging eventually as well?

kashif commented 2 years ago

yes eventually

esbraun commented 1 year ago

Update: I've empirically found significant improvement using the negative binomial distribution for a operations related count valued forecasting problem using the TFT implementation from kashif's repo over the mxnet TFT gluonts implementation and other gluonts algorithms. The implementation, however, lacks the full featured covariate support of the gluonts mxnet model. Hopefully the support for more loss functions could be wrapped into remediation for #2416.