Find the seal

[aborowicz]

Background extraction (back_extract) is functional but not particularly awesome.

Once there are more images, implement a machine learning image classification step to find seals.

[aborowicz]

Phil has adjusted some of the sea ice seg/class code to find and mask seals, so we have a random forest classification script that's consistently better than the non-ML version we'd been running. There may be some version issues with cv2. Specifically, I got errors on cv2.findContours if I didn't assign three variables to it.

We're also interested in not writing jpgs throughout to avoid compressing and recompressing along the way. Currently we reload the image and have to threshold it to clean it up post-jpg. This can be avoided by saving to an uncompressed format

[aborowicz]

It's a long process training the classifier. Large images can take up to or over an hour to segment, so I've put together a training set of smaller images. Used the output classifier to mask a bunch of seals, but it looks like I need a greater diversity of backgrounds in the training set. A not-insignificant number of seals had part of their body masked out, especially those in strange positions and those on rocky/pebble/cobble backgrounds.

I've made a new training set of small images with a more representative sample of backgrounds and will run it tomorrow morning when there is enough time to let it run.

[phil-mcdowall]

If you're having trouble with speed theres a couple of things you could try. 1) Modify the script to segment all the images before showing any for marking. The out put of the segmentation function is a labeled image so you can always save it and reload it later. That way at least you don't wait for each one. You could also load the saved segmented images back for classification rather than resegmenting them. 2) Reduce the resolution of the images before segmenting. It won't make a lot of difference to the superpixels, you'll just have to make sure the mask scales back up to the original. I'm working on the GPU based implementation of the segmentation step. I'll let you know when its running.

[hlynch]

Also, you could always use XSEDE! We have an allocation for these projects just waiting to be used. Might not speed up any one image, but it would help in processing the whole catalog.

On Mon, Aug 15, 2016 at 6:27 PM, Phil McDowall notifications@github.com wrote:

If you're having trouble with speed theres a couple of things you could try. 1) Modify the script to segment all the images before showing any for marking. The out put of the segmentation function is a labeled image so you can always save it and reload it later. That way at least you don't wait for each one. You could also load the saved segmented images back for classification rather than resegmenting them. 2) Reduce the resolution of the images before segmenting. It won't make a lot of difference to the superpixels, you'll just have to make sure the mask scales back up to the original. I'm working on the GPU based implementation of the segmentation step. I'll let you know when its running.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/polar-computing/3DSeals/issues/5#issuecomment-239948528, or mute the thread https://github.com/notifications/unsubscribe-auth/AE2UebcrxJ1qXk2mw_r-x3gh8NecPyrOks5qgOfbgaJpZM4JWMOk .

Dr. Heather J. Lynch Associate Professor of Ecology & Evolution Stony Brook University 113 Life Sciences Bldg. Stony Brook, NY 11794 Phone: 631-632-9508 Web: lynchlab.com

[aborowicz]

So after running retraining and reclassifying, I compared the outputs of the non-ML foreground extractor function (which combines cv2.grabCut with a method that boosts gamma, blurs, and then finds the longest contour in the output image) with the ML method.

There were a lot of images from both which were still not ideal, but in a simple which-is-better comparison, out of 66 images the non-ML version was better in 41 cases (61.1%) and ML was better in 26 cases (38.8%)

I'm going to go back and attempt to classify the images (by hand) as to substrate, as I think that may be an important factor here. Using the quickshift classification, I think the neighborhood of pixel values for a grey seal on grey rocks is too homogeneous. There were also some confusingly poor results for seals on snow, though I think they were mostly lighter-colored animals where perhaps the darker fur-to-lighter snow appeared more like a gradient than a hard difference.

If this is the case, maybe I'll try to implement a step that selects the foreground extraction method based on substrate, funneling some images to ML and some to the long-contour method.

Robocop:

[aborowicz]

New development: At least some of the ML failures have to do with rotation. Basically the mask is laid on top of the image, but reversed, as below. All images are rotated when they are first edited (as part of the manual preparation/ranking workflow) but clearly somewhere in the bowels of computer land there is reference to the original orientation. Perhaps these images need to be opened in editing software, rotated, and saved as a new file.