Attend presentation on deep learning in the MI1 class

[cristiprg]

A domain expert is going to give a presentation instead of a normal MI1 class on the 8th. Be present!

[jeyhunkarimov]

@cristiprg I like this format. You can write the summaries of each issue, after you solved. For example, was the presentation helpful?

[cristiprg]

@jeyhunkarimov

Lecture Ok, so the presentation was very inspiring. Youssef, the presenter, made a good introduction to what deep learning is, how it relates to neural networks and how it can solve the classification problems differently from traditional ML algorithms.

One reason people decided to add more layers to neural networks (hence "deep" networks) is that it does not overfit so easily as it the case with very wide layers (many neurons in a hidden layer). However, a deep network with many layers suffers from other problems, such as Vanishing Gradient, Random Initialization, and yet again overfitting. Luckily, the solutions to these are somehow accessible, not too difficult. One smart solution to overfitting is dropout.

There are different architectures in the literature , such as Autoencoders, Stacked Autoencoders, Restricted Boltzmann Machines and Convolutional Neural Networks. Each of them solves a slightly different problem and for classifying Lidar data, so far I believe that Convolutional NN are most suited. Therefore I will have to read more about this type of NN.

Although probably not related to classifying Lidar data, another interesting fact I learned is the possibility to transfer the model for a different task. This can lead to very interesting and surprising results.

Follow-up After the lecture, I had a short chat with Youssef on trying to classify lidar data with deep neural networks. The task is not easy and needs thorough investigation on current literature. We exchanged emails and he provided me additional tips and finally we set up a meeting on Thursday (14 Dec.) to dig "deeper" into deep neural networks.

mi_tutorial_dl_20161208.pdf