Deep learning reading group

A repository to keep track of papers in the reading group at University of Oslo.

Comments to the papers can be found under Issues.

Sessions

13 09.03.17

Papers:

• Asynchronous Methods for Deep Reinforcement Learning: https://arxiv.org/pdf/1602.01783.pdf

12 23.02.17

Papers:

• Human-level control through deep reinforcement learning: https://storage.googleapis.com/deepmind-data/assets/papers/DeepMindNature14236Paper.pdf

Additional:

• Blog by Karpathy http://karpathy.github.io/2016/05/31/rl/, though he use policy gradient instead of Q learning.
• Blog by DeepMind https://deepmind.com/blog/deep-reinforcement-learning/. Some videos from the paper, and some improvement and achievements.

11: 24.11.16

Papers:

• End-To-End Memory Networks: (https://papers.nips.cc/paper/5846-end-to-end-memory-networks.pdf)
• Hybrid computing using a neural network with dynamic external memory: (http://www.nature.com/nature/journal/v538/n7626/pdf/nature20101.pdf)

10: 13.10.16

Papers:

• Neural Turing Machines (https://arxiv.org/pdf/1410.5401v2.pdf)

Additional:

• Alex Graves talks about the Neural Turing Machines https://www.youtube.com/watch?v=_H0i0IhEO2g

9: 29.09.16

Papers:

• Generative Adversarial Nets (https://arxiv.org/pdf/1406.2661v1.pdf)
• Adversarial Autoencoders (http://arxiv.org/pdf/1511.05644v2.pdf)

8: 15.09.16

Papers:

• Variational Inference: A Review for Statisticians (https://arxiv.org/pdf/1601.00670v3.pdf)
• Tutorial on Variational Autoencoders (https://arxiv.org/pdf/1606.05908v2.pdf)

Other things discussed during the group:

• Morphing faces : http://vdumoulin.github.io/morphing_faces/
• Wavenet : https://deepmind.com/blog/wavenet-generative-model-raw-audio/

7: 01.09.16

Papers:

• Deep Kalman Filters (https://arxiv.org/pdf/1511.05121v2.pdf)

6: 11.05.16

Papers:

• Distilling the Knowledge in a Neural Network (http://arxiv.org/abs/1503.02531 and https://www.youtube.com/watch?v=EK61htlw8hY)
• Do Deep Nets Really Need to be Deep? (https://papers.nips.cc/paper/5484-do-deep-nets-really-need-to-be-deep.pdf)

Suggested readings:

• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding (http://arxiv.org/abs/1510.00149)
• SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters and <0.5MB Model Size (http://arxiv.org/pdf/1602.07360v3.pdf)

5: 27.04.16

Papers:

• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (http://arxiv.org/pdf/1506.02142v4.pdf)

Suggested reading: Dropout as a Bayesian Approximation: Appendix (http://arxiv.org/pdf/1506.02157v4.pdf)

4: 14.04.16

Papers:

• Generating Sequences With Recurrent Neural Networks (http://arxiv.org/pdf/1308.0850v5.pdf)

Suggested reading: Parts of the article is summarised in the lecture (https://www.youtube.com/watch?v=-yX1SYeDHbg)

3: 16.03.16

Papers:

• Detecting Methane Outbreaks from Time Series Data with Deep Neural Networks (http://link.springer.com/chapter/10.1007/978-3-319-25783-9_42#page-1)
• Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition (http://www.mdpi.com/1424-8220/16/1/115/html)

2: 24.02.16

Papers:

• Natural Language Understanding with Distributed Representation (http://arxiv.org/pdf/1511.07916v1.pdf)

Suggested reading: I also recommend the following two blog posts, they are both very good introductions to RNN and LSTM models.

http://colah.github.io/posts/2015-08-Understanding-LSTMs/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/

1: 12.02.16

Papers:

• A Primer on Neural Network Models for Natural Language Processing (http://u.cs.biu.ac.il/~yogo/nnlp.pdf)

Suggested reading: Natural Language Understanding with Distributed Representation (http://arxiv.org/pdf/1511.07916v1.pdf) (Ch. 1-3, 5 and perhaps 7).