Clustering images

[yhr91]

Exploring how best to apply unsupervised learning in parallel with supervised learning

[yhr91]

@smyth7 Would be really useful if you could write about your efforts so far - your plans ahead and if the rest of the team can contribute in some way

[smyth7]

Just to say that I've seen this but I'm a bit swamped this week. Below is an email I sent to Tobias a couple a weeks ago after my first look at the data. I'll spread the comments around any other relevant issues, with links to the kaggle discussions, when I get the chance.

• Some manual cleaning of the training set is required (see discussion board)
• There are many hints that the final dataset, to be released later, will involve unseen image types
• Splitting the images based on intensity (see my R code on the repo) to identify intensity groups = image types. This will allow separate model training when training data is available for given intensity group, and an unsupervised thresholding/watershed approach on test set images not in training day (perhaps needed in next round)
• Test generalization of supervised approaches on training data whose image types are not in the current test set
• Test ensembles on both (pretty obvious really)