Cluster similar images - Githubissues

JoshVarty / KaggleUtils

A collection of utilities I use for EDA, feature engineering etc.

MIT License

1 stars 1 forks source link

Cluster similar images #6

Open JoshVarty opened 5 years ago

JoshVarty commented 5 years ago

For image related tasks it might be useful to cluster similar images together. We may want to count these images across the train and test set and see whether or not the distributions are equal.

JoshVarty commented 5 years ago

Thoughts:

Train a network
Remove the head
Take the output for each image
Run PCA on each vector
Run K-Means clustering on all of our vectors

JoshVarty commented 5 years ago

It isn't working. Possibly because the features our network learns focus on "Does this image have cancer or not"

Possible workarounds

Same approach but cluster 1 and 0 separately. (Might not work for test set though)
Try either earlier layers of the network or simply use the raw input

JoshVarty commented 5 years ago

Tried with more success on the input image. Things to try:

Try without PCA
Try with different number of PCA components
Can we seed the K-means clusters?
- I think we can use init. See here