SNIC with adaptivelz normalized distance

[bobleegogogo]

Hi Moritz,

pysnic is a great piece of python package based on SNIC, many thanks for your efforts.

We are working with remote sensing images superpixel method and we have tried SLIC, SLIC-CO (zero parameters version of SLIC), and SNIC. While I wonder is there also possibility of build SNIC-CO to further reduce the parameters of the algorithms? or does this idea fit well with the original design of SNIC algorithms?

How do you think about this? Any suggestions on how one may extend the current pysnic to zero parameters version?

Best,

Hao

[MoritzWillig]

Hi Hao, thanks for your question! I also have thought of zero parameter versions for SNIC before. Overall there are two primary components we need to take care of: The distance metric and the seed initialization/refinement.

Setting aside the seed placement for now, the metrics between SLIC and SNIC are quite similar (the normalization factors for distance and color are squared in the SLIC formula). If you wish to, you can just plug in the adapting formulas. See the examples and following code for the current implementation: https://github.com/MoritzWillig/pysnic/blob/8f51232046c88b3f4ddcec7fab080e8ec8b2c189/pysnic/metric/snic.py#L24

For color normalization ASLIC uses the maximum color deviation of a superpixel from the previous iteration. One problem we face here, is that SLIC still requires an initial superpixelation to find the maximum color deviation for each cluster. As SNIC is non-iterative we need another approach to estimate the color deviation. Sampling pixels close to the initial seed and updating it throughout the run could give a sufficient results. I'm also not quite sure if taking the maximum color distance is a good heuristic for normalizing the values, since outliers, that are due to noise, will have a large impact. Normalizing with the standard deviation, or fitting some distribution should be more stable.

We also got a hyper-parameter to trade off color and image distance. As this trade-off is dependent on the given application, I'm not quite sure if we can do better than performing a parameter search. (In that context we could also try learning the distance metric directly using a neural net or similar ...)

From the examples I have worked with, I got the impression, that creating a better seed initialization may be more beneficial for the overall result, than fine-tuning the hyper-parameter.

I hope that you find some of the comments useful. If you have any further ideas or question please let me know! Best, Moritz

[bobleegogogo]

Hi Moritz,

Definitely, I find your comments very helpful. In general, I agree with you that hyper-parameter tuning could be less beneficial for improving superpixel performance than having a better seed initialization.

Indeed, the nature of SNIC for being non-iterative could play a big role when finding the maximum color deviation for each cluster, or I am also not sure is this estimation really necessary or beneficial. Give the fact that remote sensing images can be more complex and sophisticated then daily photos, a uniform _compactness factor_ could also be more intuitive if we cannot find a better way to find the local maximum. I guess I will forget the idea of zero parameter SNIC for now.

In that context we could also try learning the distance metric directly using a neural net or similar ...)

This is indeed an interesting idea, deep learning or machine learning using NN from the image to get an end-to-end superpixel segmentation sounds really cool, although not sure about the state-of-the-art research, I guess this should be unsupervised or self-supervised DL/ML approach.

Best Regards,

Hao