Perceiver IO (with a completely learnt query)

openclimatefix / power_perceiver

Machine learning experiments using the Perceiver IO model to forecast the electricity system (starting with solar)

MIT License

7 stars 1 forks source link

Closed JackKelly closed 2 years ago

JackKelly commented 2 years ago

Try:

Query for the encoder:

[x] Completely random, learnt query.
[x] Convert to python script so we can easily run several experiments at once
[x] Or, query is generated by a neural net which knows what we're trying to predict.
[x] Or, most of the query is random. But part is the same query that we'll give to the decoder.

JackKelly commented 2 years ago

OK, this is done. I couldn't get the Perceiver IO to perform quite as well as the "full attention" model I was playing with on Friday