Deep Learning Notations

by Elvis Saravia

(Part of the DAIR initiative)

Notebook: 📘 Deep Learning Notations 📘

Aim: This notebook contains useful notations widely used in deep learning papers and educational materials found online. I used similar notations used in the Deep Learning book written by Ian Goodfellow, Yoshua Bengio and Aaron Courville. I will also provide sample code using PyTorch to show the type of data structures and concepts these notation may represent.

Uses: You can reuse the notations in this notebook as a cheatsheet to assist you in writing your research papers, presentations, and blogs. It's also good resource for reviewing important mathematical notations used widely in deep learning research and other related fields. I provide example code in PyTorch but as an exercise, you can try generating similar code using Numpy or Tensorflow. (The code shouldn't be too different.)

Requirements: PyTorch

Preview:

Future: There are plenty of mathematical concepts and notations which can be added to this notebook. I will continue to add as time passes. If you are into this kind of thing, feel free to make a pull request. My hope is that it provides a useful mathematical foundation and reference for beginners in the field of machine learning and deep learning.