search

lessthanoptimal / BoofCV

Fast computer vision library for SFM, calibration, fiducials, tracking, image processing, and more.
http://boofcv.org
1.07k stars 257 forks source link

Improve Reading QR Code With Bright Spots #286

Closed WonsokYoo closed 3 years ago

WonsokYoo commented 3 years ago

Hi, I've switched from other QR code decoding library after reading (https://boofcv.org/index.php?title=Performance:QrCode) and testing.

Performance was surprisingly good even it was built by JAVA.

I've found some weired use-case but I don't understand why..

I've tried 'DetectQrCodeApp' in demonstrations.jar with an uneven illumination QR code image. The binarization result was perfect with 'LOCAL NICK'. Every bits were binarized correctly seeing with 'View: Binary' in GUI. However, part of bits which were in the dark region of the original image were read by '0' (black dots). I could see those bits that were recognized using boofcv GUI. ('Bits' checkbox).

Do you have any ideas?

Thanks in advance!

lessthanoptimal commented 3 years ago

Could you post a picture where you think it's having issues? Much easier to diagnose.

On Thu, Jul 22, 2021 at 9:50 PM WonsokYoo @.***> wrote:

Hi, I've switched from other QR code decoding library after reading (https://boofcv.org/index.php?title=Performance:QrCode) and testing.

Performance was surprisingly good even it was built by JAVA.

I've found some weired use-case but I don't understand why..

I've tried 'DetectQrCodeApp' in demonstrations.jar with an uneven illumination QR code image. The binarization result was perfect with 'LOCAL NICK'. Every bits were binarized correctly seeing with 'View: Binary' in GUI. However, part of bits which were in the dark region of the original image were read by '0' (black dots). I could see those bits that were recognized using boofcv using GUI. ('Bits' checkbox).

Do you have any ideas?

Thanks in advance!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lessthanoptimal/BoofCV/issues/286, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFUOV7VLAMNC26THIZLIFLTZDYKBANCNFSM5A3JGFQA .

-- "Now, now my good man, this is no time for making enemies." — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

WonsokYoo commented 3 years ago

@lessthanoptimal Thanks for your kind reply. Unfortunately I cannot attach images on GitHub because of my internal network situation. Could you provide your e-mail address so that i can send the image?

WonsokYoo commented 3 years ago

@lessthanoptimal Now, the e-mail seems sent properly. Thank you so much.

lessthanoptimal commented 3 years ago

Just deleted messages with the e-mail in an attempt to keep spam down.

lessthanoptimal commented 3 years ago

I was able to replicate the problem and know what's going on. The issue is that decoding works using a single threshold for the entire QR Code that's dynamically computed. This makes its performance decoupled from global thresholding that's used to find the locator patterns. The decoder doesn't use the binary image since that can have "holes" in it due to how local thresholding works and local vs global approaches for thresholding have very different artifacts.

Good news is that I think the algorithm can be improved to handle this case without too much trouble. Can I add this image to the test dataset? This would be publicly available and needs to either be released under a creative commons type license or transferred.

WonsokYoo commented 3 years ago

@lessthanoptimal Yes, you can add the image to the test dataset (By any chance, please do not mention my affiliation :)).

BTW, I didn't understood regarding the algorithm. About your mention "global thresholding that's used to find the locator patterns",
→ What if I choose "LOCAL" thresholding method? Is it still find locator "GLOBALLY" independent from the thresholding method what I've chosen ?

Also, you said that "decoding works using a single threshold for the entire QR Code that's dynamically computed.", "The decoder doesn't use the binary image" → Then, what's the purpose for binarization? How did you dynamically compute a single threshold? Is the threshold affected by threholding method such as LOCAL_NICK?

lessthanoptimal commented 3 years ago

yeah after I wrote that I thought it might be confusion.

Steps

  1. Threshold (e.g. LOCAL_NICK) applied to entire image to create a binary image
  2. Square patterns are found inside of binary image
  3. Sets of 3 squares are found and a candidate list of QR Codes is made
  4. For each candidate QR Code it computes a new threshold specifically for that QR code by looking at the pixel intensity around each square
  5. The QR code specific intensity is then used to read bits

I've simplified things a bit but that's the general idea.

What I think is happening here is that the bright spot it skewing the QR code specific threshold so that the dark region (which isn't even considered when computing the QR code specific threshold since it uses the square regions) can't be read.

Going to attempt to generate more images for the test set, but if you had any more where it fails that would be helpful.

WonsokYoo commented 3 years ago

@lessthanoptimal Thanks for your clear explanation. Now I understood how it works. Please let me know if you have any progress :)

lessthanoptimal commented 3 years ago

First pass with a new thresholding algorithm is done. A threshold is now estimated in the lower-right corner of the QR code. Then for each bit a threshold is selected by doing bilinear interpolation between the 4-corner thresholds.

Need to do more testing to make sure there isn't a major performance hit, but so far things look good. Improved overall detection by 1%, with a lot of that coming from the "bright_spots" test.

https://github.com/lessthanoptimal/BoofCV/tree/feature/qr_lighting

lessthanoptimal commented 3 years ago

https://github.com/lessthanoptimal/BoofCV/pull/289

If you want to take a look at the fix there's the PR. It should get merged soon. Only a very small speed it. You will still need to change it to LOCAL_NICK or some other local method for your use case. The current default works better overall, but LOCAL methods seem to handle bright spots much better.

WonsokYoo commented 3 years ago

@lessthanoptimal Thanks for your support. I tested on my PC today and now it works! Obviously, the performance seems better than the previous one. I will try more samples and if any issues occur I will report to you. Thanks again!

lessthanoptimal commented 3 years ago

Based on my internal tests with "bright spots" detecting finder patterns is the biggest weakness. If speed isn't a issue you can improve that running the detector multiple times with different magic numbers for local region size in the binarization step.

Going to close this issue for now, but feel free to open a new one if decoding is still a problem on a lot of images.