martinapugliese
commented
3 years ago From the notebook I had on this:
Using transfer learning means using a pre-trained network and add to it or fine-tune it to your data. A typical pre-trained network has been released by someone after having being trained on some data and might be applied to your problem with some edits.
The idea is to run the pre-trained network on your data in order to extract features from it, then passing them to additional layers you add to customise the whole model to your problem.
A typical example is in image classification: a dataset like ImageNet has (and is) used for training networks over the classification of what an image displays, it's a dataset with 1000 classes and millions of images. If your problem is that of classifying images, and your classes are among the ImageNet ones, you can use a pre-trained network to make it recognise the features over your data and then customise.
TODO add example in Keras
Existing networks
TODO restructure this all notebook
TODO imagenet conest, ILSVCR
training times and beast of GPUs used increase in time, fetch some graphs
LeNet
Y LeCun 1998
AlexNet
by Alex Krizhevsky & , &, started it all. 2010? Won the ImageNet contest in 2012, which is the year when CNNs were first shown to be great for the task.
TOdo on imagenet and history
ZF Net
from 2012 on, everybody started submitting solutions with CNNs. ZF Net (Z F surnames) won in 2013. fine tunes AlexNet . uses the concept of deconvolutional network
VGG Net
2014, wasn't winner. simple
GoogleNet
2015, inception module (it's not sequential and was among first nets to destroy the tradition that cnns had to be sequential), won in 2014.
ResNet
Microsoft 2015. very deep, won 2015.
R-CNN
region-based cnns. faster
Generative adversarial nets
2014,
generating image descriptions
cnn + rnn
spatial transformer net
2015
and put a link to in the obj detection with ANN page