an advice for ch1

[yebangyu]

Backpropagation is really an important and fundamental topic in deep learning

yeah, i admit that bp is a little math-heavy and a little hard for new guys

but i also can not imagine that a guy who can not understand bp can undestand deep learning, really.

bp hurts people but it is a good hurt and a must hurt.

you can not omit it . so , please add bp in ch1.

[pmayd]

You don't need bp to do deep learning or even understand it, so not important at all, only if you want to understand certain parameters or implement then fundamentals on your own, but you definitely don't need it to apply any deep learning algorithm

[yebangyu]

can not agree with @pmayd

if someone does not understand bp, he will be confused with code like 'value.backward()' in page 21 of the book. what is backward ? why back ? back what?

The book is titled with ‘inside deep learning’, inside means internals, means 'good at' ,means 'proficient'.

sure, we don't need it to apply any deep learning algorithm but if we do not understand it, we are just the guys who can only call some apis of pytorch

[EdwardRaff]

The book covers a lot of what's not covered in other books - particularly around combining mathematical intuition to code in one place. That is where people are writing their own code and implementing their ideas, so that's where the book focused. It's also an intersection that most other books don't really tackle. They either do all the math or all the code, but not much tying them together. I hope the book has done a good job of being a middle ground between theory & practice.

Most people using deep learning are never going to implement the backward function though, and having taught ML and DL to tons of people over the years, starting with details on BP drives away a lot of people's early ones. It's very intimidating to start with, but with modern frameworks, also unnecessary to start with. They can start writing code and getting experience without knowing how it works under the hood - just as many people learn to write programming without memory allocators, TLBs, paging, and all these other details that have become so ubiquitous and transparent that it doesn't require an explicit introduction.

I think backprop is almost at that point, and I don't think I have anything new to say about BP that hasn't been said better by others. That's why I pointed readers to Grokking Deep Learning for a more detailed introduction to BP. I would be writing a second and third book if I put in everything I'd love to talk about into it - and my editor would have killed me if I tried to make the book even longer again - it's one of the longest Manning books!

But again, its a book meant to be the start, not the destination. Maybe if you have some backpropagation blogs/other books you really like, this could be a good thread for folks wanting to go deeper on that detail on where to look next?

[benjwolff]

I think it's a wise decision not to go over BP in this book. But if you want to learn more about BP and how it is computed, I recommend this lecture.