2.8 From the image, extract the test strip and its coloured reagens patches

[mtwestra]

We need to extract the location of the strip, and determine the colours. From the software, we should know how many reagens patches we are expecting.

The steps to follow: 1) determine the location and orientation of the strip 2) cut out and straigthen the strip 3) determine the location and order of the reagens patches. This can also be used as a quality check, as the strips usually are asymetrical (the reagens patches are placed on one side). 4) determine the colours of the patches.

We should investigate the effect of the patches being wet on the perceived colour.

[ldporsius]

@mtwestra Though what you propose is undoubtedly a sound approach, maybe we can simplify things. I am thinking: using OpenCV, we can detect the color patches directly, skipping steps 1 and 2. We 'know' -based on measurements of the design- where the (black) area is where the strip lies, from previous steps (see issue 5 and 8). The base color of the strip is white (is that always the case?). So what we can do is: in the black area search for area's that are not black or white, in other words, have color. Step 3: the quality check, I do not understand what you mean, please explain. The order can be based on the x and y coordinates of the rectangles found that contain a certain color. Step 4: determine the colours. Could we take the average color like in existing Caddisfly code?

[mtwestra]

Yes, we can certainly try this. We will need the order, but that won't be a problem. For the average color, we can use the caddisfly code: in a circle, determine the color that most often occurs.

On 22 Jul 2015, at 16:59, Linda Porsius notifications@github.com wrote:

@mtwestra Though what you propose is undoubtedly a sound approach, maybe we can simplify things. I am thinking: using OpenCV, we can detect the color patches directly, skipping steps 1 and 2. We 'know' where the (black) area is where the strip lies, from previous steps (see issue 5 and 8). The base color of the strip is white (is that always the case?). So what we can do is: in the black area search for area's that are not black or white, in other words, have color. Step 3: the quality check, I do not understand what you mean, please explain. The order can be based on the x and y coordinates of the rectangles found that contain a certain color. Step 4: determine the colours. Could we take the average color like in existing Caddisfly code?

— Reply to this email directly or view it on GitHub.

[ldporsius]

I tried the approach I explained previously, but I found it was not very reliable. More often than not, very bleak patches are not found. Today I tried something else, more along the steps 1) and 2) -find the location of the strip -cut out the strip

It does not work when taking a picture through a sheet of plastic, but otherwise this is the result. From here we could do several things:

1) straighten strip and base detection on known values about the strip in question (where we know the patches are on this particular brand of strip) 2) try and detect color in this sub-image

The steps I take sofar are:

• make a gray-scale image.
• apply a treshold. Everything above 40 gets white, below 40 gets black.
• Canny edgedetection. This results in a binary image.
• find contours in the Canny-result-image. I use RETR_TREE and CHAIN_APPROX_SIMPLE as parameters. RETR_TREE results in an hierarchy where contours that lie within a contour get numbered from 0 to highest level continuously. (the third value of the hierarchy array contains the value for parent). CHAIN_APPROX_SIMPLE results in getting only the corners of the contour.
• since our design is a contour of the strip within the contour of a black area, I loop over the contours until I find the one that has the highest value for parent. That should be the innermost contour, hence our strip.
• I check in this loop if contour area is large enough (>250pix), to filter out noise.

I feel I do a bad job explaining contours, but this tutorial makes it very clear:

http://opencvpython.blogspot.ca/2013/01/contours-5-hierarchy.html

[mtwestra]

Sounds like a good approach, lets try it with a real strip

On 30 Jul 2015, at 17:29, Linda Porsius notifications@github.com wrote:

I tried the approach I explained previously, but I found it was not very reliable. More often than not, very bleak patches are not found. Today I tried something else, more along the steps 1) and 2) -find the location of the strip -cut out the strip

It does not work when taking a picture through a sheet of plastic, but otherwise this is the result. From here we could do several things:

1) straighten strip and base detection on known values about the strip in question (where we know the patches are on this particular brand of strip) 2) try and detect color in this sub-image

The steps I take sofar are:

make a gray-scale image. apply a treshold. Everything above 40 gets white, below 40 gets black. Canny edgedetection. This results in a binary image. find contours in the Canny-result-image. I use RETR_TREE and CHAIN_APPROX_SIMPLE as parameters. RETR_TREE results in an hierarchy where contours that lie within a contour get numbered from 0 to highest level continuously. (the third value of the hierarchy array contains the value for parent). CHAIN_APPROX_SIMPLE results in getting only the corners of the contour. since our design is a contour of the strip within the contour of a black area, I loop over the contours until I find the one that has the highest value for parent. That should be the innermost contour, hence our strip. I check in this loop if contour area is large enough (>250pix), to filter out noise. I feel I do a bad job explaining contours, but this tutorial makes it very clear:

http://opencvpython.blogspot.ca/2013/01/contours-5-hierarchy.html

— Reply to this email directly or view it on GitHub.

[ldporsius]

I have made some improvements in strip detection and finding the patches.

• first I convert the strip-area to a Lab image.
• I treshold the L-channel to make a black/white image
• I find contours within the altered L-channel This results I think in a more reliable strip detection.

Then I tried two methods to detect the patches: 1) using a OTSU treshold 2) using a adaptive treshold Both work with a Lab image.

Though OTSU seems more simple, for now I get better results with adaptive treshold. OTSU does not detect the bleakest color in my teststrip, whereas adaptive does. The result with 2)

Also, it varies with quality of exposure, but I have good hopes that based upon an already calibrated image, the results will be better.

BTW: on the right you see the mean value of each patch.

[ldporsius]

Test with OTSU. This is an 'inbetween' image, that shows where to look for color in the original image. You can see that three of the four are very clearly detected, but the very light pink patch not at all.

[ldporsius]

http://opencvpython.blogspot.ca/2013/05/thresholding.html

This article explains about tresholding.

[mtwestra]

This looks very good indeed!

On 4 Aug 2015, at 15:26, Linda Porsius notifications@github.com wrote:

I have made some improvements in strip detection and finding the patches.

first I convert the strip-area to a Lab image. I treshold the L-channel to make a black/white image I find contours within the altered L-channel This results I think in a more reliable strip detection. Then I tried two methods to detect the patches: 1) using a OTSU treshold 2) using a adaptive treshold Both work with a Lab image.

Though OTSU seems more simple, for now I get better results with adaptive treshold. OTSU does not detect the bleakest color in my teststrip, whereas adaptive does. The result with 2)

Also, it varies with quality of exposure, but I have good hopes that based upon an already calibrated image, the results will be better.

— Reply to this email directly or view it on GitHub.

[ldporsius]

I tried something else still. Detection of patches based on difference between minimum value of chroma in strip and chroma of each pixel. So if pixel has a chroma higher than minimum + 5, it means it is part of a color patch.

Applying a treshold that sets the pixels to black if meeting the criteria. The result of this:

And the original:

[mtwestra]

After a lot of experimentation, the conclusion here was that directly searching for color patches is not a feasible method. The main problems are: 1) Sometimes the patches hardly have any colour at all, ie, when they are close the '0 ppm' value 2) Often, the patches cast small shadows, which complicate the contour finding.

An example of a difficult case showing both problems is shown below:

The new method that will be implemented is to directly use the geometry and layout of the strip and the patches. This means: 1) Measure the strip layout: length and width of total strip, central position of each patch, and width of each patch. 2) find the complete strip by means of thresholding + contour finding 3) rotate and clip the strip area 4) from the knowledge of the strip layout, determine the position of the patches.