Ideas

[alexklibisz]

Some ideas for improvement, ordered loosely by "bang for the effort buck":

• Down-size the images (e.g. 100x100) to make training faster and decrease the number of weights. It's worth trying to see if we lose much (if anything) in terms of performance.
• Create a smaller training dataset by taking a representative sample from the full training dataset. Uses this to check if a model will converge before taking much longer to train on the full dataset.
• Try a single dense output layer (similar to most of the image classification networks). I'm hesitant to do this because it will have to be a sigmoid activation (instead of softmax) which is hard to train. Adding batch normalization to this might be a good option.
• Try each of these architectures, all available with Keras: http://www.pyimagesearch.com/2017/03/20/imagenet-vggnet-resnet-inception-xception-keras/
• Cost function that explicitly weights the false negative errors more than false positive errors in correspondence with the F-Beta evaluation metric on Kaggle.
• Investigate which classes are being missed.
• Abstract the train/predict setup to a "driver" file that also takes the model as a CLI parameter. This would allow us to modify things more quickly and decrease the surface area for bugs. Would have to assume a common interface for each model... this sounds familiar.
• Concatenate output from early layers with output from later layers before the dense classifiers. See UNet: https://arxiv.org/abs/1505.04597, FCNs for Semantic Segmentation: https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf.
• Squeezenet: https://arxiv.org/abs/1602.07360
• Multi-scale networks: http://ieeexplore.ieee.org/iel5/6022827/6033131/06033589.pdf

[alexklibisz]

Papers covering multi-class classification (architecture, loss function, etc.)

• Here is a paper from IJCNN having to do with error measures and optimization for multi-class classification. 076_0283.pdf
• https://arxiv.org/pdf/1406.5726.pdf
• https://arxiv.org/pdf/1312.4894.pdf

[alexklibisz]

• Consider adding some completely negative images (e.g. images that belong to none of the classes) to force the network to learn to ignore irrelevant features.

• Use a confusion matrix to express the co-occurrences of true labels and predicted labels on the training set. This might help narrow down common error cases.

[alexklibisz]

Moved these ideas to a wiki page to organize a little better: https://github.com/alexklibisz/kaggle-planet/wiki/Ideas