Scanned QR Code is not recognized

[mjeanrichard]

Hi,

I am having trouble with decoding the following QR code. The Code has been printed and then scanned again and can then no be read anymore by the ZXing library. I mostly tried with the CommandLineDecoder and the WindowsFormsDemo from the Clients folder. I tried with all kinds of different settings but none seem to work.

What do I need to do to make this work?

Thanks for your help.

[micjahn]

The error correction fails but I have no idea why. The java-based online decoder fails too: https://zxing.org/w/decode.jspx What's the content of the code and which error correction level did you choose?

[mjeanrichard]

It has correction level M (15%). It was gernerated from here: https://qr-rechnung.net/ It is basically the new standard for creating bills in switzerland. The Data contained within the above QR code is

SPC
0200
1
CH5204835012345671000
S
Max Muster & Söhne
Musterstrasse
123
8000
Seldwyla
CH

1949.75
CHF
S
Sarah Beispiel
Musterstrasse
1
8000
Seldwyla
CH
NON

Auftrag vom 21.2.2022
EPD
//S1/11/220221/30/102673386/31/220221/32/7.7/40/0:30

It is interesting, that I can actually scan the code above with my phone (shown on the github page) and it can decode it (differen app though)...

[jmgraf]

I think error correction might be failing due to the reduced contrast between the QR code and the white background of the QR code itself and the QR being varying shades of gray, leading to increased background noise. Interestingly it looks as if the QR code has been inserted into its containing document, since the background colors vary somewhat.

I wonder if the phone app is applying some binarization or sharpening to increase contrast to the QR code that ZXing isn't performing to be able to recognize the code.

You could attempt pre-processing the QR before sending it to ZXing, although I have spent some time working on a very similar QR code to the one you posted and was only able to get marginal increase in performance over the standard implementation.

[bartdk-be]

• We are having a similar issue.
• Both Aspose and inliteresearch (ClearImage) are able to recognize the QR code, but ZXing is not
• Are there any plans to improve the recognition ? Or a way we can tweak the settings ?

[manwithsteelnerves]

@mjeanrichard Is your dev platform Unity-Android? If so, can you try scanning your barcodes with this apk which is built with Easy ML Kit?

[mjeanrichard]

@manwithsteelnerves No, my platform is windows (.net core with WPF)

[axxel]

I'm the maintainer of https://github.com/zxing-cpp/zxing-cpp and was curious whether my QRDetector reimplementation would fare any better than the original java version. Unfortunately, it does not. The problem with this particular sample is not the shading/contrast/noise but a slight deformation/distortion that leads to the sampling grid being around 1 module off near the center of the symbol. The way to fix this is to incorporate the alignment patterns, which is exactly their purpose. Unfortunately neither zxing nor zxing-cpp not (presumably) ZXing.Net does this.

[n-exo]

Did you find any solution for this ? We're struggling with the same topic. Scanned swiss-qr-codes are not recognized.

[mjeanrichard]

I am building my own. Works well so far, but not yet ready to publish. Will report back as soon as I have it done.

[n-exo]

Would really appreciate

[MrHuang77]

Have you solved the problem? I failed to identify some distorted QR codes with zxing

[mjeanrichard]

@n-exo Do you have by any chance a sample of a QR that you cannot read? So that I could check if it is read correctly and use it in my tests...

[n-exo]

@mjeanrichard Sure! thankful for any help with that 👍

[kltye]

I'm the maintainer of https://github.com/zxing-cpp/zxing-cpp and was curious whether my QRDetector reimplementation would fare any better than the original java version. Unfortunately, it does not. The problem with this particular sample is not the shading/contrast/noise but a slight deformation/distortion that leads to the sampling grid being around 1 module off near the center of the symbol. The way to fix this is to incorporate the alignment patterns, which is exactly their purpose. Unfortunately neither zxing nor zxing-cpp not (presumably) ZXing.Net does this.

Hi,

I was wondering if there'd be any chance you'd be able to port those changes to the ZXing.Net library? I have the attached QR code that isn't recognized by the .Net library (and the Java one, for that matter), but works fine with yours.

Alternatively, @micjahn, any ideas why this image doesn't work? Thank you both!

[n-exo]

@mjeanrichard Do you had any success reading these damaged QR-Codes in the meanwhile ?

[axxel]

@kltye I just noticed that your last comment was potentially intended to be addressed to me. My answer is: I can't / won't. I'm not a .NET developer and busy improving the c++ port. Said improvements will result in the ability to scan the OPs Swiss code example with zxing-cpp.

[mjeanrichard]

@n-exo I just now got to the point where I can fully decode a QR Code. I tried with your image. The problem seems to be that has a very low resolution. This makes it very sensitive to a not perfect detection of the alignment pattern. I might be able to improve upon it but I am not sure. What helps a lot though, is just scaling the image up (I tried with 500%):

The following images are debug output generated by my decoder. You can see that it detected the alignment marks fine, but because the calculated center is off a little, it doesn't get the modules to the lower right correctly (and possibly others).

The scaled up version is perfect though (I did nothing special, just used paint.net and scaled it):

So the question is, why are your images of such a low resolution and can you do something about it? Maybe just scale it up before processing it?

@kltye I can confirm the suspicion about the alignment mark. My Decoder uses the lowest right mark and can decode your sample fine.

[kltye]

@kltye I can confirm the suspicion about the alignment mark. My Decoder uses the lowest right mark and can decode your sample fine. @mjeanrichard Excellent! I have mostly solved my issue by scanning at a slightly higher resolution. Most importantly however, I invert the image colour before processing it, and for some reason that seems to have fixed it with ZXing.Net. But I'd love to try your library when it's ready for use - if only for a backup if ZXing fails on other use cases.

[axxel]

As a bit of a followup to my comment above I'd like to mention that release 2.1 of https://github.com/zxing-cpp/zxing-cpp has an improved QRDetector that does now use the mentioned alignment patterns and can handle large distorted symbols like the first one above or even a lot worse ones like e.g. this one.

[buddycat75]

Hello, As I try to find an issue to trouble decoding Swiss QRCode (since days...)

--> I scan approx 50 Swiss Invoice to 300 dpi, color per day --> Mostly, the botom left pattern of the QR Code cause the problem (if you just make some manual colorisation of the botom left pattern, everytings become ok)

My solution (95% of my QR invoices are well decoded) --> first attemp --> crop (botom part of the page) + reduce image by 0.5 + TRY_HARDER + TRY_INVERSE (works almost always and fast) --> seconde attemp --> full page + no reduction + TRY_HARDER + TRY_INVERSE --> last attemp --> crop (botom part of the page) + enlarge image by 2 + TRY_HARDER + TRY_INVERSE

Not realy "nice" solution, but it works pretty well.

[n-exo]

Hi People,

For you guys having problems decoding badly scanned Swiss QR-Slips- I was able to get a success-rate of almost 100% by resizing the QR-Code if Failed. Pretty simple by multiplying height and width:

[Chris21x]

@n-exo Thanks for it. Is your "pdfImage" a whole page or a cropped region of it? No memory problems multiplying a whole page by 5? And: did you convert it to BW or greyscale before?

[axxel]

@kltye / everyone: There is a bit of an update to my answer above from a year ago: zxing-cpp now comes with a .NET wrapper.

[qoo91812211228]

@n-exo Hi, I tried this method but it doesn't work for me. Some images can be multiplied by 5 to read barcodes/qrcodes, some cannot. Any suggestions to help me do better?

[Chris21x]

@n-exo Hi, I tried this method but it doesn't work for me. Some images can be multiplied by 5 to read barcodes/qrcodes, some cannot. Any suggestions to help me do better?

Hey you both @qoo91812211228 : I actually do have a quite extensive probable solution to your problem. I wanted to post this for weeks now, will do it over Easter Holidays, is that early enough for you, qoo?

I literally ran tens of thousands of test pages (1'400 per single test, approx. 30 different tests each of it with this solution and the commercial one) with various settings, with image manipulations like brightness, sharpening, blur, b/w-conversion, magnification, resolution settings etc. But it's quite a work to write a summary - but I found the best way for me and our company. Recognition 112% of the highly professional commercial package.

Is Easter Weekend early enough?

Kind greetz from Switzerland - Chris

[axxel]

@Chris21x interesting results with your 112% comparison. May I ask which commercial library you are referring to?

Also, I would be very curious to learn how your solution fares compared to https://www.nuget.org/packages/ZXingCpp/0.2.1-alpha. maybe that would be a relatively simple addition to your testing setup? See also https://github.com/axxel/zxing-bench.

[Chris21x]

@axxel We're providing quite a complex and both graphically and scriptable customizable package of document management (only a small part of it is QR/Barcode reading). We went away from Abbyy Finereader used to, both OCR and QR/Barcode reading. Target was to get rid of limitations (no. of pages per month) and connected costs of the commercial package (and the subjection to sudden 'policy changes' of Companies which urges us to not only change the whole package (and its 37 sub-projects) but also all rolled-out customer solutions). I developed it to a Tesseract plus ZXing (plus - spoiler! - a second QR Code package) and I managed to increase Recognition Quality, Speed AND Costs for our solution/customers.

Thanks for your hint concerning the Alpha! Gimme time over the Easter Holidays. I really do want to share the experience - maybe it helps and another developer need not to run through these painful tests with 30 x 1'400 pages each (not even counting the processing time, surveillance, analytics)

Therefore I want to give a HUGE Thanks to @buddycat75 whose post above inspired me in the first place to move into this direction

Kind Greetz - Chris

[Chris21x]

Post 1 / 3

Hint: Don*t want to bother or care about details? My resulting solution proposal can be found in post number 3 / 3 below.

Dear all I want to share my path to an “optimum” solution – for us & from my perception – as announced above. I try to keep this summary brief, all questions are welcome! This elaboration will spread over several posts as it breaks boundaries – I apologize. I will move from “overview” to “more details”.

Our Use Case: We provide a software package that is used to analyze complex documents concerning content (OCR), layout analysis, storage within a DMS and filling in needed meta information – and thus it includes also: QR/Barcode recognition. Customers usually run upto 3k documents a day. We’re reading and interpreting all kinds of QR/Barcodes, but this anlysis focuses on “SwissQR” for which we actually added an extra “mode”. Means: we use this method below, which is slower, only when “directed to do so” – to maintain our overall processing speed.

Our Test Sample for each test:

• Used ZXing Version: ZXing.Net package by Michael Jahn Version 0.16.9
• Sample of 700 “real life” documents with (all together) 1’400 pages
• Various numbers of pages per document, from 1 to 11
• According to (OCR) analysis, they contain 704+ (max. 709) SwissQR codes, some of it more than one
• Quality:
• • ca. 85-90% of it is scans: a mix of b/w, greyscale, colour, 200dpi (maybe 5%), mainly 300dpi
• • ca. 10-15% are eMail attachments.
• • ca. 15+% of the scans are distorted, dirty or otherwise in a “bad shape”
• • All PDFs are rendered before processing

Our Benchmark: The benchmark is our previously used commercial package (not known to be cheap)

Our prioritized Goals:

1. Recognition rate of SwissQR (our referenced benchmark is the commercial package which I defined as the basis, thus 100%, for the analysis)
2. Processing speed of SwissQR containing pages
3. Processing speed overall (average, with or without SwissQR and Non-SwissQR codes present)

Our Measurement: Each result of a test run is separately analyzed and benchmarked and it shows exactly the “additional number of SwissQR recognized” in addition to the default mode we use

Our Approach:

• If the directive is “default QR/Barcode” we do a single run with ZXing.net with the flag TRY_HARDER set to true
• Directive “SwissQR”: we iterate through the default recognition (see above), then upto FIVE other “fallbacks” if no SwissQR code is found so far

The target of these posts is to describe the method with the five selected fallbacks and also, which methods did NOT bring ANY additional benefits.

[Chris21x]

Post 2 / 3

Remark If I say “cropped” below, I mean I crop the document to the lower part of a page according to the official specification (see there). Most of the documents follow these rules, but if a Swiss “Einzahlungsschein” is added to an invoice separately and they’re scanned manually, people tend to scan them on the top of the scanner glass. It means: the “cropped methods” do not detect them, they’re out of bounds. Therefore I’m always adding a full screen analysis on top as a potential fallback.

EXCLUDED / Dismissed Methods: An important part for a successful solution is to exclude and dismiss certain methods without any benefit. Excluded methods of image manipulation

• Enlarging to 500% (zero additional recognition): both, cropped image to lower part acc. to specs or non-cropped (full page) image
• Full page with 8bpp B/W conversion (zero additional recognition): SwissQR is by specification B/W
• Full page with indexed greyscale conversion with default black threshold (128) (zero additional recognition)
• Full page with Gaussian Blur with different lengths and weights (also tested with augmented contrast and sharpening afterwards) - zero additional recognition
• Various test runs with adjustment of brightness and/or contrast - zero additional recognition
• ZXing.Net switch “TRY_INVERSE” – never any difference Some of the above is logical: SwissQR by specification is B/W

How-to read Results of chosen Procedures: As mentioned: Benchmark is a commercial package and is defined as “100%”. 700 documents, 1’400 pages, according to full text analysis 704 upto max. 709 SwissQR in them. Processing time and hit rate of every solution / (additional) fallback step is measuread and analyzed. Absolute processing time is not relevant here as our use case, I think, is more complex. Maybe the delta can be relevant to some of you.

Results of the Benchmark:

1. Benchmark: Commercial package recognized 638 (out of actually 704+ SwissQR Codes): defined as 100%
2. ZXing.Net: Run “default”: Full page, no image manipulation, solely set “TRY_HARDER” Recognized 536 (out of actually 704+ SwissQR Codes: 84.0% of benchmark)
3. Final solution (details see below) with a combination of methods PLUS a second QR Decoder: Recognized 703 (out of actually 704+ SwissQR Codes: 110.2% of benchmark)

[Chris21x]

Post 3 / 3

Our Solution Approach:

• We used a second Software packed to fill in the gaps of recognition in some of the “fallback steps”.
• “QRCodeDecoderLibrary” as a NuGet-Package by (unknown) Version 2.1.0 This seems to me to be a bit inaccurate and we don’t use it except for these additional dedicated fallback steps. It also seems to be a bit slower in processing time compared to ZXing.Net
• What I do say: I validated all recognized SwissQR Codes read by this package within the fallback options: they’re all valid.

Hint: We’re talking (partly) of image processing, make sure your memory processing, maintenance and clearing covers all cases!

Our actual resulting Solution: The order of the steps focus on “max. additional hit rates” considering also the overall processing time.

1. Read full page with ZXING without image processing with the switch “TRY_HARDER” Result here: 583 out of 704+ (84.0% of benchmark)
2. Read full page with “QRCodeDecoderLibrary” Result here: additional 97 SwissQR codes were found (adding +15.2% of benchmark)
3. Read CROPPED bottom part of the page (acc. to specification) with ZXING, shrink image by 50% with the switch “TRY_HARDER” Result here: additional 20 SwissQR codes were found (adding +2.9% of benchmark)
4. Read full page with “QRCodeDecoderLibrary”, shrink image by 50% Result here: additional 27 SwissQR codes were found (adding +4.2% of benchmark)
5. Read CROPPED bottom part of the page (acc. to specification) with ZXING, enlarge image to 200% with the switch “TRY_HARDER” Result here: additional 12 SwissQR codes were found (adding +1.9% of benchmark)
6. Read full page with ZXING, shrink image by 50% with the switch “TRY_HARDER” Result here: additional 11 SwissQR codes were found (adding +1.7% of benchmark)

Questions are welcome. I hope I could help other coding friends who face such a challenge. Personal comment: The SwissQR is crap by definition. A selected medium redundancy (15%) built in and already occupying half of this by the “Swiss Cross” and its border in the middle: this is near to “bound to fail” in the real life in the first place.

Kind regards from Switzerland - Chris

[axxel]

@Chris21x Thanks for the detailed explanation of your approach and your results. To lure you into giving ZXingCpp a try, I will now boldly claim that ZXingCpp with a single pass will surpass your 110% results and do that 2 to 10x faster that what you currently use. I'll happily accept defeat if that should not be true ;).

[Chris21x]

@axxel Fair enough - Challenge accepted. I do LOVE such things. Especially as the goal is to potentially give everyone around additional know-how and information. Last but not least: additional information to you, the developer. I thought about a reasonable approach:

1. To give you a fair shot, we need to get a broader document set / base of examples (Remember? 703 out of maybe 704, maybe only a few more, SwissQR were read) - it could also be, that there is only ONE additional in the documents and that one really is corrupt - then you would have no chance to superceed. I'm thinking of 2'000+ documents. I do have another sample set of 17k documents for my OCR stress tests but these are unclassified and the most part is no invoice...)
2. In order not to create too much work here to adopt the whole of our Client/Server Solution (surely would take some days), I would run on this expanded & different (!) document basis:
   • The processing with our package using the commercial solution (defining this as 100% "benchmark")
   • The processing with our above described solution
   • The processing with a dedicated piece of software using basically your (expanded) example on your GitHub page

The result would be a pure comparison of the recognition rates. Therefore in this first test batch we can't compare processing time/speed (But IF you're successful, in a following step we would check this in a next dedicated test).

What do you think?

What I do want to ask you: to be able to create a real comparison, can you adjust your example code (which reads all kinds of barcodes) to a dedicated one for SwissQR incl. switches like TRY_HARDER? I can do this as well but it would take more time. I'm a bit short of time currently...

Conditions for counter-challenge approved?

:-D

PS: Your source code framework for this challenge should be a simple cmd accepting the file name including path of a single PDF. Ideally: Create a ZIP Package including a VS Project.

[axxel]

Using the sample set you already have is perfectly fine with me. I did not realize that those 110% are basically 100% of all detectable symbols. Using a different one that poses the same challenges as in the OP is also fine. As for limiting the search space, the sample code already support that, you simply have to pass "QRCode" as the last parameter ('barcode-format-list'). tryHarder is enabled by default and should stay on.

ZXingCpp does not operate on PDFs, only on pixel data. But so does ZXing.NET if I'm not mistaken. I have no idea how to canonically render a PDF into a bitmap using .NET infrastructure. If you need help with how to construct an ImageView object from your pixel data, let me know what kind of image/buffer data structure you are working with.

[Chris21x]

That's fine, I will use the same PDF renderer and settings as we do in our package. This will now take some time, as I said, I'm currently a bit short on time.

[Chris21x]

@axxel - Our Challenge Just as interim information: I chose and prepared the testing document basis: the test document set is kinda "worst case":

• Invoices of 2021 (first year of obligatory SwissQR within Switzerland)
• By that time: probably worst quality (overwhelmingly scanned, minority mail attachments, SwissQR Codes partly not conform with the spec., ect.)
• Number of Documents: 6'392 - Number of pages: yet unknown
• Number of effectively existing SwissQR within these documents: will not be evaluated; thus ist a pure "relative benchmarkung", relative to the commercial solution rated/set as 100%, not an absolute one.

Will proceed as follows, starting this evening:

1. Run these say 6.4k documents through the commercial package (takes roughly 12+ hrs) & analyze This result is used as benchmark = 100%
2. Run these documents through our solution described above & analyze
3. Run these documents through a "mock-up" with your alpha-Package & analyze

Will take surely a week or two - has to run during the night(s). Processing time will not be analyzed - not comparable. (Would be next step).

Kind greetz from Switzerland - Chris

PS: Is here any way to send a direct message to you?

[axxel]

@Chris21x thanks for the update. You can reach me under zxingcpp at gmail dot com.