You might, if you have time, consider adding in associative memory to the structure of your neural networks on the presumption that it will give them extra "powers".
Associative memory was a serious topic in the 1960's (see archive.org) through the 1970's and up to the time of Hopfield.
Unfortunately all those researchers missed a very simple simple algorithm for doing associative memory:
https://devtalk.nvidia.com/default/topic/1023786/cuda-programming-and-performance/walsh-hadamard-transform-based-ai/post/5208577/#5208577
The idea is to provide the deep network with "soft" memory that it can actually learn to use.
You might, if you have time, consider adding in associative memory to the structure of your neural networks on the presumption that it will give them extra "powers". Associative memory was a serious topic in the 1960's (see archive.org) through the 1970's and up to the time of Hopfield.
Unfortunately all those researchers missed a very simple simple algorithm for doing associative memory: https://devtalk.nvidia.com/default/topic/1023786/cuda-programming-and-performance/walsh-hadamard-transform-based-ai/post/5208577/#5208577 The idea is to provide the deep network with "soft" memory that it can actually learn to use.