Deep Learning for Coders with fastai and PyTorch

[vvksh]

https://learning.oreilly.com/library/view/deep-learning-for/9781492045519/

Also check out the interview of author: jeremy howard on lex fridman show

=> I will read the book in two passes; first scan through all chapters quickly, then will go through and do exercises.

I already read through chapter 1 sometime ago which introuces the fastai library; will start from chapter 2.

total of 19 chapters; expecting to finish first scan in a week or so, say Jan 15; will do the second pass with exercises depending on how much bandwidth I have; will probably be a long tail.

For the exercises, I am using notebooks (with gpu from paperspace.com); tried to setup google cloud notebooks with frustrating experience from unclear error about quotas and zones.

Progress:

• [x] chapter 1
• [x] chapter 2
• [x] chapter 3
• [x] chapter 4
• [x] chapter 5
• [x] chapter 6
• [x] chapter 7
• [x] chapter 8
• [x] chapter 9
• [x] chapter 10
• [x] chapter 11
• [x] chapter 12
• [x] chapter 13
• [x] chapter 14
• [x] chapter 15
• [x] chapter 16
• [x] chapter 17
• [x] chapter 18
• [x] chapter 19

[vvksh]

chapter2: end-to-end description of developing, deploying a DL solution; recommended using Viola to turn notebooks into webapps (local server) and binder to turn notebook into a deployed webapp using github.

• also touched on problem of out-of-domain data and data shift being issues when deployed in production.

chapter 3 is on ethics; quickly skimmed; nothing new; talks about bias and other issues in systems used in society (facial recognition, ad targeting etc)

[vvksh]

chaper 4: gives you a general feeling of what a nn is; some warm up using pytorch to create a simple nn; and intuition regarding how SGD works.

[vvksh]

chapter 5: practical overview of handling images for training, cross-entropy loss, picking learning rate, freezing layers, applying different learning rates, how much to train. etc.

[vvksh]

chapter 6: not much, how to forumulate single/multiple classification or regression

[vvksh]

chapter 7: label smoothing and mixup to train state of the art model

[vvksh]

chapter 8: collaborative filtering; using users and example, can use nn to discover latent factors (type of content) and biases( whether the movie is good etc); showed how to create collab filtering using intuitive way and then using deep nn way.

[vvksh]

chapter 9: didnt read; intro to sklearn, traditional ML etc

[vvksh]

chapter 10: language models, RNN; didnt read

[vvksh]

chapter 11: how to write transforms chapter 12: RNNs, LSTMs, how to do it in pytorch

[vvksh]

chapter 13: barely read; intro to what a convolution is and how to do CNN in fastai and pytorch chapter 14: resnets

[vvksh]

chapter 15: DL architectures for vision/NLP , also touches on what to do when overfitting happens chapter 16: SGD+optimizers+how callbacks feature in fastai helps add several ideas into the training loop elegantly.

[vvksh]

chapter 17: basics of neural network [skipped] chapter 18: model interpretation; shows how to use pytorch's hook to implement CAM (class activation map) (lets you see which part of image lead to decision making)

[vvksh]

chapter 19: shows how to implement a fastai learner with all pieces, decent code, didnt go through it.

[vvksh]

verdict: more info on theory using fastai library as facilitator that I expected; definitely learned about dataloaders, pytorch modules, pytorch hooks etc which could be useful.