search

depthfirstlearning / depthfirstlearning.com

Other
196 stars 29 forks source link

Add curriculum for Neural Mechanics paper #44

Closed ramasesh closed 3 years ago

ramasesh commented 3 years ago

I'd like to add a curriculum for the recent paper Neural mechanics: symmetry and broken conservation laws in deep learning dynamics. There is a lot of depth to this paper, and the payoff (being able to calculate exact training dynamics of certain parameter combinations) makes that worth studying. Like the Resurrecting the Sigmoid curriculum, I'd probably structure it to be mostly based around problem sets, except with more interactive (coding/jupyter notebook) problems. If this sounds good, I can get started.

cinjon commented 3 years ago

That sounds amazing. How can we help?

ramasesh commented 3 years ago

Sorry, I missed this. I'll submit a pull request with an outline soon, and any eyes on that would be very helpful!

cinjon commented 3 years ago

Sounds good!