2.2 Capture sharp image when calibration code is found

[mtwestra]

We need to check ourselves if the preview image the camera has captured is sharp. For this, we need to create a focus measure, based on the captured images.

It is likely that before we use these methods, we need to correct the overall illumination of the image by normalising the grayscale image to the (0,255) value range. We might also need to use a gaussian filter first, to reduce noise.

There are diffent ways to handle this: 1) using the normalised variance This method computes the normalised variance of (part of) a grayscale image. For example, we could use the finder pattern part of the image, for which we know the location.

The variance is defined as: (1/N mu) * SUM (p(i,j) - mu)^2

where mu is the mean of the grayscale image, and p(i,j) is the grayscale value of a pixel. If there is more contrast in the image, this value will be larger.

2) using the laplace filter In this case, we apply a laplace filter over the image. A laplace filter compares nearby pixels and computes the rate of change: the larger the contrast between two nearby pixels, the larger this value.

After the laplace operation, there are two possibilities: a) find the maximum value. This is the easiest, but might be susceptible to noise, because a single noise pixel with a high contrast value will dominate the results. We should probably try this first. b) compute the value where 90% of the values falls in. For a low-focus image, most differences between the pixels are small, so 90% of the values will lower than some value. For a high-focus image, there will be more differences that are large, so the 90% border will be at a higher value. This is shown in this image: http://stackoverflow.com/questions/15839928/how-to-detect-when-an-image-is-out-of-focus

We will try both and compare the results.

The code will need to: 1) as fast as possible, determine if an image is in focus or not. For this, we might want to cut out the 3 finder patterns, and determine the focus only on those parts. As these have the highest contrast (black/white). 2) be insensitive to the global illumination level, so determine focus in both light and dark images

To test this, we need to create some test patters and determine the efficiency of the method. This can be done in photoshop.

In addition, we should capture and store some 100 images, and determine the focus visually.

[ldporsius]

Questions:

"cut out the 3 finder patterns". What do you mean by that? The location of the finder patterns is only known after a successful capturing of them. Or we should try and find a way to locate them in a live preview?

" be insensitive to the global illumination level, so determine focus in both light and dark images". Why would that be important? I don't see how the focus has to do with the brightness of the image.

[ldporsius]

I could try and make a histogram of a live preview and work with the 90% treshold. What would be the value of the treshold? The other two options we tried.

[mtwestra]

The idea was to take 'sub-images', which only has the finder patterns in them. In that way, the laplace operator only needs to act on a small image, so it will be very fast.

The exposure needs to be correct, because if it is not, it will be very hard to define the proper treshold for the laplace data.

On 5 Jul 2015, at 16:31, Linda Porsius notifications@github.com wrote:

Questions:

"cut out the 3 finder patterns". What do you mean by that? The location of the finder patterns is only known after a successful capturing of them. Or we should try and find a way to locate them in a live preview?

" be insensitive to the global illumination level, so determine focus in both light and dark images". Why would that be important? I don't see how the focus has to do with the brightness of the image.

— Reply to this email directly or view it on GitHub.

[mtwestra]

The idea here would be that: 1) make a greyscale image (of the finder pattern parts) 2) do the laplace operation 3) make the histogram of the result 4) find the location of the 90% treshold.

We'll need to see what a proper treshold is by looking at the result of different images.

On 5 Jul 2015, at 16:33, Linda Porsius notifications@github.com wrote:

I could try and make a histogram of a live preview and work with the 90% treshold. What would be the value of the treshold? The other two options we tried.

— Reply to this email directly or view it on GitHub.

[ldporsius]

[ldporsius]

By applying a bilateral filter I get the above result. It is a bit improved don't you think? But the colors are still much bleaker than the original.

[mtwestra]

What is a bilateral filter?

I think the first thing to do now is improve the focus. m

On 9 Jul 2015, at 14:11, Linda Porsius notifications@github.com wrote:

By applying a bilateral filter I get the above result. It is a bit improved don't you think? But the colors are still much bleaker than the original.

— Reply to this email directly or view it on GitHub.

[ldporsius]

http://docs.opencv.org/modules/imgproc/doc/filtering.html

On 07/09/2015 02:59 PM, Mark Westra wrote:

What is a bilateral filter?

I think the first thing to do now is improve the focus. m

On 9 Jul 2015, at 14:11, Linda Porsius notifications@github.com wrote:

By applying a bilateral filter I get the above result. It is a bit improved don't you think? But the colors are still much bleaker than the original.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/akvo/caddisfly-app/issues/5#issuecomment-119953811.

[ldporsius]

This issue seems the right one to talk about our latest results in focusing the camera. The problem we had, is that focusing the camera interferes with the automatic exposure an Android camera instance does. We tested this again and again, and always with the same outcome: when focusing exposure is lost. The idea behind our solution is: maybe it is a matter of time. Give the camera enough time to adjust its exposure, and only focus when needed. Now, when is it needed? We trust that if exposure is alright (not over lighted, not completely in the dark) the zxing code is able to find the finder patterns. That is already a check on the quality of the exposure, so we use that. If finder patterns are found, then it is the time to focus the camera. We do not need to do it continuously. But, there is still a problem with that, because we focus the camera in a callback (onPreviewFrame) that is continuously called upon. So if we rest at this, we still get a continuous focusing and the problem remains. Therefore, in the callback class, we introduced a static property, we called it 'firstTime', which is a boolean, to be set to true if it is the first time we try to focus the camera. The callback class is only instantiated once, to be sure, so this static should hold whether the class has focused or not. Now, if finder patterns are found, and it is the 'firstTime', we focus the camera, and then return. Nothing happens, besides we have now a focused camera. Then the 'second time' is called immediately after that (or so it should be). If 'second time', the camera does not focus, but goes on with the rest of the code, cut out the part between finder patterns, do the perspective warp, etc.

A flaw is, that it is possible that between 'first time' and 'second time' the position of the camera changes. We should monitor if 'second time' indeed happens fast enough to make this reliable.

It is necessary to make this work if one wants to take a new picture, to set the static boolean 'firstTime' to true in the onPause of the activity (that instantiates the callback class). That seems to work ok.