holoviz-topics / EarthSim

Tools for working with and visualizing environmental simulations.
https://earthsim.holoviz.org
BSD 3-Clause "New" or "Revised" License
65 stars 21 forks source link

Discussion - Grabcuts default guess #204

Closed kcpevey closed 5 years ago

kcpevey commented 6 years ago

I was just brainstorming this morning and it seems like there might be a way to let GrabCuts make a semi-intelligent automated first guess for coastline delineation. I think one reason that GrabCuts works so well is there tends to be a fairly dramatic distinction between land and water. Admittedly, I have minimal knowledge of image processing and how GrabCuts actually works, but... is there a way that we could sample the RGB values for the whole image, create a spectrum of the results, grab the two main peaks and feed the RGB values representing those peaks to grabcuts? This would only work if the image has a large distinction between land and water and if the colors are fairly uniform across each type.

Examples where I think that would produce reasonable first guesses:
lejeune

atch

Might work less well for color-varying waters:
scs

jbednar commented 6 years ago

I'm sure that something like that could be done, and probably has been done by someone. To decide which of the two main groups is water, you can probably use some measure of variance -- sometimes the water is darker than the land, sometimes lighter (as in this middle picture), but in each case (due to mixing) the water is more spatially uniform. So once there are two groups, you could run a local spatial homogeneity index on it to guess which is water. If both groups are homogenous, as in some desert areas, I guess you could pick the group closer to blue as the water one.

In any case, is getting this initial guess the limiting step in your workflow? For hydrology modelling it seems like later steps still need a good bit of manual cleanup and tweaking, so getting a good initial guess may not be worth going to a lot of effort and complexity.

jbednar commented 6 years ago

If you did want to try this, we have an example of doing clustering on a satellite image to form n groups by pixel similarity in EarthML.

kcpevey commented 6 years ago

is getting this initial guess the limiting step in your workflow?

Short answer - no, not at all. My mind was just wandering as I sat through 3 school dropoff lines... I don't really think there is any way around manual tweaking so I don't think it would be productive to put much effort into it. Also, given that it takes so long to run, adding that to spinup time would probably not be wise. The main reason why it came it is that one of intentions is to make the workflows fully automated or as close as possible. This would just be a step in that direction.

kcpevey commented 5 years ago

Closing this until the push for automation forces some implementation