ml-jku / hopfield-layers

Hopfield Networks is All You Need
https://ml-jku.github.io/hopfield-layers/
Other
1.69k stars 189 forks source link

Using this for audio prediction #16

Closed robclouth closed 2 years ago

robclouth commented 3 years ago

I need to be able to predict 100 steps into the future of an audio stream. I have an N x M tensor that is analysed audio, where N is the total length in frames, and M the number of mels per frame. I would like to use a hopfield layer to predict the next 100 frames from a single input frame.

I could frame it like the simpsons image retrieval task and have the 100 x M blocks of frames as the patterns, and the M input mels as the state, but it appears that the state and pattern have to be the same size.

What's the best way to do this? Is this even the right tool for the job or am I going to quickly hit the storage limit? It's for about 10 mins of audio so N will be about 100000.

bschaefl commented 2 years ago

Hi @robclouth,

it is best to contact the authors directly via email in case of general question about this topic.