Tool classify the output data

[aleksandrmelnyk]

Sorry, may not relevant issue. But for curiosity sake, is there any tool to classify output of this tool?

[aleksandrmelnyk]

Ok, kmeans clustering can be used to classify data eg. by average.

[cbalint13]

@mamayoleksandr ,

First, thank you for your interest in this little tool !

• The goal of this tool is to relax the vast amount of pixels of large satellite / aerial materials thus making some advanced algorithm tractable with very large scale of information.
• The tool just produce the raw superpixels/segments carrying (i believe) all possible/inherited information derived from original pixels. I plan some little tool to produce graph information too (encoded into sort of hdf5) in the case someone also need the relationship/adjacency between superpixels (e.g. one want to do graph based inference like MRF, CRF, etc).
• There are many way to classify these outputs ranging from simple kmeans as you mentioned to SVM, CRF, MRF (graph-cuts), or more advanced deep learning algorithms that can be fed with ease with these superpixels insted of pixels. But such classifier task exceed the goal of this tool.

I personally had very good time relaxing pixels with this tool and reuse the information in some graph-cut (MRF) and also some mixed deep learning algorithms like [1] or [2]. I plan to release some of such ideas in form of some tools / tutorials.

• http://arxiv.org/abs/1506.07271
• http://arxiv.org/abs/1603.07063

[aleksandrmelnyk]

@cbalint13 , thanks for the response. Yep, quite a variety of classifying approaches. For me quite useful is kmeans clustering by 1_average, 2_average, 3_average, x(centroid) and y(centroid).

[aleksandrmelnyk]

Hey Cristian,

thanks for those links, looks very promising, but sorry for unrelevant question, don't you know of any implementations of Deep Boltzmann Machine, or conditional random field algorithms which could classify these superpixels? I was using kmeans, random forest and multinominal regression but all provide not that very good results, eg some noise present, recognised objects include superpixels from not related areas etc. which is logically because they analyze only colour (1_average, 2_average and 3_average), but not also relation to adjacent superpixels.

Also with pixels it is relatively easy to build graph (since you know location of pixel in grid), but how to do with superpixel..? find nearest ones pretty expensive operation.

thanks much,

Alex.

[cbalint13]

On Thu, Jun 2, 2016 at 10:11 PM, Аleksandr Melnyk notifications@github.com wrote:

Hey Cristian,

thanks for those links, looks very promising, but sorry for unrelevant question, don't you know of any implementations of Deep Boltzmann Machine, or conditional random field algorithms which could classify these superpixels?

• For DBN and RBM experiments i used: https://github.com/nitishsrivastava/deepnet [the only one good quality implementation].

I was using kmeans, random forest and multinominal regression but all provide not that very good results, eg some noise present, recognised objects include superpixels from not related areas etc. which is logically because they analyze only colour (1_average, 2_average and 3_average), but not also relation to adjacent superpixels.

• Use 1_stdev, 2_stdev, 3_stdev too ! So use all 6 dimensional property of the superpixel in your favorite classifier. BTW, stddev is just as important as the color information, it reflects the variance of color within the superpixels. E.g a low std_dev reflect that the superpixel is compact (e.g water surface, black road), a higher one reflect that is full with tiny textured variances (e.g leaves, grass, tree corona). For RBM too i use all 6 dimensions (like it would be hyperspectral thing).

Also with pixels it is relatively easy to build graph (since you know location of pixel in grid), but how to do with superpixel..? find nearest ones pretty expensive operation.

• And why can't build graph with superpixels ? It is easy with superpixels too. I use a std::set<std<pair<A,B>> and traverse whole segmented image (pixel by pixel) and do an .insert() when adjiacency flap occours (lookup all 4 neighbours always). So in std::set<> i will end with the full graph, also the duplicate edge/links are discarded. See: http://openrisc.rdsor.ro/graph.cpp code snippet.
• In case of doing graph over the pixels you end up with 4 connected symmetrical one and with a big amount of graph that e.g. no CRF will be able to tract it ever. In case of superpixels graph you end up with a bit asymmetrically connected graph but a much more relaxed one. MRF, CRF and others can takle assym graphs. There is no information loss switching from pixel to superpixel ! The weights relationship within the graph can be high dimensional, can be composed by ratio of arrays and difference of stddev and/or color so can be expressed in many ways. Or can play a dual graphs with relationships on each other, one graph play on color energy second graph play on stddev and so on possibilities are limitless with superpixels.

Balint Cristian e-mail: cristian.balint@gmail.com

[aleksandrmelnyk]

Cristian,

thanks much for the response, pretty helpful.