CSCI181: Deep Learning

About the Instructor

About the Course

This is a course on deep learning (not big data).

Course Objectives:

Learning objectives:

1. Write basic PyTorch applications
2. Understand the "classic" deep network architectures
3. Use existing models in a "reasonable" way
4. Understand the limitations of deep learning
5. Read research papers published in deep learning
6. Understand what graduate school in machine learning is like
7. (Joke) Understand that Schmidhuber invented machine learning

My personal goal:

1. Find students to conduct research with me

Expected Background:

Necessary:

1. Basic Python programming
2. Linear algebra
3. Calc III
4. Statistics

Good to have:

1. Machine learning / data mining
2. Lots of math
3. Familiarity with Unix and github

Resources:

Textbook:

1. The Deep Learning Book, by Ian Goodfellow and Yoshua Bengio and Aaron Courville; I will assume that you already know all of Part I of this book (basically the equivalent of a data mining/machine learning course)
2. Various papers/webpages as listed below

Deep learning examples:

1. Images / Video

   1. Deoldify
   2. style transfer
   3. more style transfer
   4. dance coreography
   5. StyleGAN
   6. DeepPrivacy
   7. https://thispersondoesnotexist.com/
   8. https://thiscatdoesnotexist.com/
   9. Deep fakes
   10. In Event of Moon Disaster

2. Text

   1. Image captioning
   2. AI Dungeon
   3. https://www.thisstorydoesnotexist.com/
   4. https://translate.google.com

3. Games

   1. AlphaGo
   2. Dota 2
   3. StarCraft 2
   4. MarioCart
   5. Mario

4. Other

   1. iSketchNFill
   2. scrying-pen
   3. Tacotron

The good:

1. most influential research in 2019 is deep learning papers
2. /r/machinelearning
3. recent open source AI programs
4. The state of jobs in deep learning
5. The decade in review

The bad:

1. Machine learning reproducibility crisis
2. Logistic regression vs deep learning in aftershock prediction
3. Pictures of black people
4. NLP Clever Hans BERT
5. Ex-Baidu researcher denies cheating at machine learning competition

Computing resources:

1. Google Colab provides 12 hours of free GPUs in a Jupyter notebook
2. Kaggle provides 30 hours of free GPU
3. I have a 40CPU/8GPU machine that you can access for the course
4. I have another 4CPU/1GPU machine that needs someone to set it up

Videos:

1. 3blue1brown
2. 2 minute papers
3. arxiv insights

Schedule

Assignments

There are no exams in this course.

Late Work Policy:

You lose 10% on the assignment for each day late. If you have extenuating circumstances, contact me in advance of the due date and I may extend the due date for you.

Collaboration Policy:

You are encouraged to work together with other students on all assignments and use any online resources. Learning the course material is your responsibility, and so do whatever collaboration will help you learn the material.

Accommodations for Disabilities

I want you to succeed and I'll make every effort to ensure that you can. If you need any accommodations, please ask.

If you have already established accommodations with Disability Services at CMC, please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course. You can start this conversation by forwarding me your accommodation letter. If you have not yet established accommodations through Disability Services, but have a temporary health condition or permanent disability (conditions include but are not limited to: mental health, attention-related, learning, vision, hearing, physical or health), you are encouraged to contact Assistant Dean for Disability Services & Academic Success, Kari Rood, at disabilityservices@cmc.edu to ask questions and/or begin the process. General information and the Request for Accommodations form can be found at the CMC DOS Disability Service’s website. Please note that arrangements must be made with advance notice in order to access the reasonable accommodations. You are able to request accommodations from CMC Disability Services at any point in the semester. Be mindful that this process may take some time to complete and accommodations are not retroactive. It is important to Claremont McKenna College to create inclusive and accessible learning environments consistent with federal and state law. If you are not a CMC student, please connect with the Disability Services Coordinator on your campus regarding a similar process.