search

whitead / dmol-book

Deep learning for molecules and materials book
https://dmol.pub
Other
606 stars 120 forks source link

Chapter 6 -- Linear systems #140

Closed d3e5560 closed 2 years ago

d3e5560 commented 2 years ago

"The main advice I would give to beginners in deep learning are to focus less on the neurological inspired language (i.e., connections between neurons), and instead view deep learning as a series of linear algebra operations where many of the matrices are filled with adjustable parameters. "

Deep learning is distinctly not a series of linear operations! That was tried decades ago before the realization that multiple layers of linear operators is still... linear.

Deep learning's power comes from the nonlinearity and composiiton of functions, e.g.,

Output =w_4 f(w_3g(w_2h(w_1z(x+x_0)+z_0)+h_0)+g_0)+f_0

Most of those functions (g,h,w) are distinctly nonlinear! At any rate, it is the combination of nonlinearity and composition of functions that brings the power to the approach.

whitead commented 2 years ago

Hi @d3e5560 thanks for the feedback. I've revised the sentence to be clear that a neural network also includes non-linearities in commit 48108ccdc4885404236a1fc6410094677cc23050