Optimization for Training Deep Models - Deep Learning Book - Githubissues

nishnik / Paper-Leaf

Contains the description of various papers I have read or reading

12 stars 0 forks source link

Optimization for Training Deep Models - Deep Learning Book #10

Open nishnik opened 5 years ago

nishnik commented 5 years ago

8.1.3 Batch and Minibatch Algorithms

8.2.1 Ill-Conditioning

8.2.3 Plateaus, Saddle Points and Other Flat Regions

8.2.4 Cliﬀs and Exploding Gradients

So we use gradient clipping

8.2.5 Long-Term Dependencies