openclimatefix / power_perceiver

Machine learning experiments using the Perceiver IO model to forecast the electricity system (starting with solar)
MIT License
7 stars 1 forks source link

Use Transformer-XL approach to attend to many hours of recent history #179

Open JackKelly opened 2 years ago

JackKelly commented 2 years ago

https://arxiv.org/pdf/1901.02860.pdf

I haven't fully understood the approach yet! After skim-reading, it feels more complex than Perceiver. So maybe stick to perceiver for now?!