Neural Tangent Kernel

[irregular-rhomboid]

NTK theory describes the behavior of neural networks in the infinite width limit as they undergo training.

Some resources that could be covered

• The original paper by Jacot et al. https://arxiv.org/abs/1806.07572
• Radford Neal's Prior for Infinite Networks paper https://link.springer.com/chapter/10.1007/978-1-4612-0745-0_2 (I have a non Springer copy, somewhere but I can't find the original link)
• Related papers in the currently blooming NTK literature

This also leads to the Tensor Programs papers and the Maximum Update parametrization, which leads to Random Matrix Theory #3 .

I'd be willing to do sessions on a couple of those, since it's very close to my research (and it involves a healthy amount of functional analysis), but I'm happy to share.