Similarities between DFT and Deep Learning

[AlexanderMath]

List of similarities between DFT and Deep Learning

1. Both solve optimization problem
2. Both require initialization techniques (e.g. minao init or xavier init).
3. Both utilize momentum techniques (e.g. DIIS or Adam)
4. The tensor that goes through the computational graph gets transformed x = x + f(x) (e.g. density mixing or residual connections)
5. Both use mixed precision (e.g. terachem keeps large ERIs in f64 and the remainder in f32, torch has automatic mixed precision with torch.amp)
6. Both uses gradients (e.g. manually implemented derivatives for forces or automatically with torch.grad)