Super pixels / unsupervised clustering

[pksorensen]

Are there any implementation already that would help solve the following problem.

Given a sample set of images (fixed size n x n pixels) would it be possible to run a unsupervised clustering to find similar super pixels / clusters.

And then run it against two images to assign clusters to each pixel, and compare two images against eachother by looking at which pixels that changed cluster.

How complicated would it be to set such flow up with chainercv for a noob like me ?

[yuyu2172]

Maybe you can work with feature maps of typical CNNs to cluster segments of pixels. Something like this. https://arxiv.org/pdf/1606.03558.pdf

VGG16 can be used for this purpose.

model = VGG16()
 # By default, __call__ returns a probability score (after Softmax).
model.pick = 'conv5_3'
conv5_3 = model(imgs)

[pksorensen]

My images have 10+ bands of different sensors, so i would assume that I need to build a custom model and not use a pretrained one. Most of them is based on RGB bands of random images i guess.

Would like to use sentinel 2 satellite images as examples. They consist of 13 bands. 4 at 10m, 6 at 20m and 3 at 60m.

Do you have any recommendation for a starting point => samples / tutorials that would help me getting started for this.

I assume that I would need to solve the following things:

• [ ] Learn how to create a sample set / data container layout. I am thinking 256x256 at 10m, 128x128 at 20m and 42x42 for the 60m bands. A sample is then 365740 uint8 values. Is this to big?
• [ ] Learn how to train feature maps The featuremaps will the be cluster representations and I should select N feature maps. Do I just select 100 ? 1000 and if the number is to high there will just be feature maps that are similar? Since we want to learn feature maps, i just need to generate enough random samles from data until feature maps converge?
• [ ] Learn what layers needs to be put after the feature maps to create clusters.

This also would form as a good basis of creating a classification if there was learning labels but that is next on my topic.