micjahn / ZXing.Net

.Net port of the original java-based barcode reader and generator library zxing
Apache License 2.0
2.74k stars 666 forks source link

Scanned QR Code is not recognized #408

Open mjeanrichard opened 2 years ago

mjeanrichard commented 2 years ago

Hi,

I am having trouble with decoding the following QR code. The Code has been printed and then scanned again and can then no be read anymore by the ZXing library. I mostly tried with the CommandLineDecoder and the WindowsFormsDemo from the Clients folder. I tried with all kinds of different settings but none seem to work. rechnung3

What do I need to do to make this work?

Thanks for your help.

micjahn commented 2 years ago

The error correction fails but I have no idea why. The java-based online decoder fails too: https://zxing.org/w/decode.jspx What's the content of the code and which error correction level did you choose?

mjeanrichard commented 2 years ago

It has correction level M (15%). It was gernerated from here: https://qr-rechnung.net/ It is basically the new standard for creating bills in switzerland. The Data contained within the above QR code is

SPC
0200
1
CH5204835012345671000
S
Max Muster & Söhne
Musterstrasse
123
8000
Seldwyla
CH

1949.75
CHF
S
Sarah Beispiel
Musterstrasse
1
8000
Seldwyla
CH
NON

Auftrag vom 21.2.2022
EPD
//S1/11/220221/30/102673386/31/220221/32/7.7/40/0:30

It is interesting, that I can actually scan the code above with my phone (shown on the github page) and it can decode it (differen app though)...

jmgraf commented 2 years ago

I think error correction might be failing due to the reduced contrast between the QR code and the white background of the QR code itself and the QR being varying shades of gray, leading to increased background noise. Interestingly it looks as if the QR code has been inserted into its containing document, since the background colors vary somewhat.

I wonder if the phone app is applying some binarization or sharpening to increase contrast to the QR code that ZXing isn't performing to be able to recognize the code.

You could attempt pre-processing the QR before sending it to ZXing, although I have spent some time working on a very similar QR code to the one you posted and was only able to get marginal increase in performance over the standard implementation.

bartdk-be commented 2 years ago
manwithsteelnerves commented 2 years ago

@mjeanrichard Is your dev platform Unity-Android? If so, can you try scanning your barcodes with this apk which is built with Easy ML Kit?

mjeanrichard commented 2 years ago

@manwithsteelnerves No, my platform is windows (.net core with WPF)

axxel commented 2 years ago

I'm the maintainer of https://github.com/zxing-cpp/zxing-cpp and was curious whether my QRDetector reimplementation would fare any better than the original java version. Unfortunately, it does not. The problem with this particular sample is not the shading/contrast/noise but a slight deformation/distortion that leads to the sampling grid being around 1 module off near the center of the symbol. The way to fix this is to incorporate the alignment patterns, which is exactly their purpose. Unfortunately neither zxing nor zxing-cpp not (presumably) ZXing.Net does this.

n-exo commented 2 years ago

Did you find any solution for this ? We're struggling with the same topic. Scanned swiss-qr-codes are not recognized.

mjeanrichard commented 2 years ago

I am building my own. Works well so far, but not yet ready to publish. Will report back as soon as I have it done.

n-exo commented 2 years ago

Would really appreciate

MrHuang77 commented 2 years ago

Have you solved the problem? I failed to identify some distorted QR codes with zxing test7

mjeanrichard commented 2 years ago

@n-exo Do you have by any chance a sample of a QR that you cannot read? So that I could check if it is read correctly and use it in my tests...

n-exo commented 2 years ago

@mjeanrichard Sure! thankful for any help with that 👍

image

kltye commented 2 years ago

I'm the maintainer of https://github.com/zxing-cpp/zxing-cpp and was curious whether my QRDetector reimplementation would fare any better than the original java version. Unfortunately, it does not. The problem with this particular sample is not the shading/contrast/noise but a slight deformation/distortion that leads to the sampling grid being around 1 module off near the center of the symbol. The way to fix this is to incorporate the alignment patterns, which is exactly their purpose. Unfortunately neither zxing nor zxing-cpp not (presumably) ZXing.Net does this.

Hi,

I was wondering if there'd be any chance you'd be able to port those changes to the ZXing.Net library? I have the attached QR code that isn't recognized by the .Net library (and the Java one, for that matter), but works fine with yours.

Alternatively, @micjahn, any ideas why this image doesn't work? Thank you both!

arrr (2)

n-exo commented 1 year ago

@mjeanrichard Do you had any success reading these damaged QR-Codes in the meanwhile ?

axxel commented 1 year ago

@kltye I just noticed that your last comment was potentially intended to be addressed to me. My answer is: I can't / won't. I'm not a .NET developer and busy improving the c++ port. Said improvements will result in the ability to scan the OPs Swiss code example with zxing-cpp.

mjeanrichard commented 1 year ago

@n-exo I just now got to the point where I can fully decode a QR Code. I tried with your image. The problem seems to be that has a very low resolution. This makes it very sensitive to a not perfect detection of the alignment pattern. I might be able to improve upon it but I am not sure. What helps a lot though, is just scaling the image up (I tried with 500%):

The following images are debug output generated by my decoder. You can see that it detected the alignment marks fine, but because the calculated center is off a little, it doesn't get the modules to the lower right correctly (and possibly others).

Original

The scaled up version is perfect though (I did nothing special, just used paint.net and scaled it):

ScaledUpBy5

So the question is, why are your images of such a low resolution and can you do something about it? Maybe just scale it up before processing it?

@kltye I can confirm the suspicion about the alignment mark. My Decoder uses the lowest right mark and can decode your sample fine.

kltye commented 1 year ago

@kltye I can confirm the suspicion about the alignment mark. My Decoder uses the lowest right mark and can decode your sample fine. @mjeanrichard Excellent! I have mostly solved my issue by scanning at a slightly higher resolution. Most importantly however, I invert the image colour before processing it, and for some reason that seems to have fixed it with ZXing.Net. But I'd love to try your library when it's ready for use - if only for a backup if ZXing fails on other use cases.

axxel commented 1 year ago

As a bit of a followup to my comment above I'd like to mention that release 2.1 of https://github.com/zxing-cpp/zxing-cpp has an improved QRDetector that does now use the mentioned alignment patterns and can handle large distorted symbols like the first one above or even a lot worse ones like e.g. this one.

buddycat75 commented 1 year ago

Hello, As I try to find an issue to trouble decoding Swiss QRCode (since days...)

--> I scan approx 50 Swiss Invoice to 300 dpi, color per day --> Mostly, the botom left pattern of the QR Code cause the problem (if you just make some manual colorisation of the botom left pattern, everytings become ok)

My solution (95% of my QR invoices are well decoded) --> first attemp --> crop (botom part of the page) + reduce image by 0.5 + TRY_HARDER + TRY_INVERSE (works almost always and fast) --> seconde attemp --> full page + no reduction + TRY_HARDER + TRY_INVERSE --> last attemp --> crop (botom part of the page) + enlarge image by 2 + TRY_HARDER + TRY_INVERSE

Not realy "nice" solution, but it works pretty well.

n-exo commented 1 year ago

Hi People,

For you guys having problems decoding badly scanned Swiss QR-Slips- I was able to get a success-rate of almost 100% by resizing the QR-Code if Failed. Pretty simple by multiplying height and width:

image

Chris21x commented 11 months ago

@n-exo Thanks for it. Is your "pdfImage" a whole page or a cropped region of it? No memory problems multiplying a whole page by 5? And: did you convert it to BW or greyscale before?

axxel commented 9 months ago

@kltye / everyone: There is a bit of an update to my answer above from a year ago: zxing-cpp now comes with a .NET wrapper.

qoo91812211228 commented 8 months ago

@n-exo Hi, I tried this method but it doesn't work for me. Some images can be multiplied by 5 to read barcodes/qrcodes, some cannot. Any suggestions to help me do better?

Chris21x commented 8 months ago

@n-exo Hi, I tried this method but it doesn't work for me. Some images can be multiplied by 5 to read barcodes/qrcodes, some cannot. Any suggestions to help me do better?

Hey you both @qoo91812211228 : I actually do have a quite extensive probable solution to your problem. I wanted to post this for weeks now, will do it over Easter Holidays, is that early enough for you, qoo?

I literally ran tens of thousands of test pages (1'400 per single test, approx. 30 different tests each of it with this solution and the commercial one) with various settings, with image manipulations like brightness, sharpening, blur, b/w-conversion, magnification, resolution settings etc. But it's quite a work to write a summary - but I found the best way for me and our company. Recognition 112% of the highly professional commercial package.

Is Easter Weekend early enough?

Kind greetz from Switzerland - Chris

axxel commented 8 months ago

@Chris21x interesting results with your 112% comparison. May I ask which commercial library you are referring to?

Also, I would be very curious to learn how your solution fares compared to https://www.nuget.org/packages/ZXingCpp/0.2.1-alpha. maybe that would be a relatively simple addition to your testing setup? See also https://github.com/axxel/zxing-bench.

Chris21x commented 8 months ago

@axxel We're providing quite a complex and both graphically and scriptable customizable package of document management (only a small part of it is QR/Barcode reading). We went away from Abbyy Finereader used to, both OCR and QR/Barcode reading. Target was to get rid of limitations (no. of pages per month) and connected costs of the commercial package (and the subjection to sudden 'policy changes' of Companies which urges us to not only change the whole package (and its 37 sub-projects) but also all rolled-out customer solutions). I developed it to a Tesseract plus ZXing (plus - spoiler! - a second QR Code package) and I managed to increase Recognition Quality, Speed AND Costs for our solution/customers.

Thanks for your hint concerning the Alpha! Gimme time over the Easter Holidays. I really do want to share the experience - maybe it helps and another developer need not to run through these painful tests with 30 x 1'400 pages each (not even counting the processing time, surveillance, analytics)

Therefore I want to give a HUGE Thanks to @buddycat75 whose post above inspired me in the first place to move into this direction

Kind Greetz - Chris

Chris21x commented 7 months ago

Post 1 / 3

Hint: Don*t want to bother or care about details? My resulting solution proposal can be found in post number 3 / 3 below.

Dear all I want to share my path to an “optimum” solution – for us & from my perception – as announced above. I try to keep this summary brief, all questions are welcome! This elaboration will spread over several posts as it breaks boundaries – I apologize. I will move from “overview” to “more details”.

Our Use Case: We provide a software package that is used to analyze complex documents concerning content (OCR), layout analysis, storage within a DMS and filling in needed meta information – and thus it includes also: QR/Barcode recognition. Customers usually run upto 3k documents a day. We’re reading and interpreting all kinds of QR/Barcodes, but this anlysis focuses on “SwissQR” for which we actually added an extra “mode”. Means: we use this method below, which is slower, only when “directed to do so” – to maintain our overall processing speed.

Our Test Sample for each test:

Our Benchmark: The benchmark is our previously used commercial package (not known to be cheap)

Our prioritized Goals:

  1. Recognition rate of SwissQR (our referenced benchmark is the commercial package which I defined as the basis, thus 100%, for the analysis)
  2. Processing speed of SwissQR containing pages
  3. Processing speed overall (average, with or without SwissQR and Non-SwissQR codes present)

Our Measurement: Each result of a test run is separately analyzed and benchmarked and it shows exactly the “additional number of SwissQR recognized” in addition to the default mode we use

Our Approach:

The target of these posts is to describe the method with the five selected fallbacks and also, which methods did NOT bring ANY additional benefits.

Chris21x commented 7 months ago

Post 2 / 3

Remark If I say “cropped” below, I mean I crop the document to the lower part of a page according to the official specification (see there). Most of the documents follow these rules, but if a Swiss “Einzahlungsschein” is added to an invoice separately and they’re scanned manually, people tend to scan them on the top of the scanner glass. It means: the “cropped methods” do not detect them, they’re out of bounds. Therefore I’m always adding a full screen analysis on top as a potential fallback.

EXCLUDED / Dismissed Methods: An important part for a successful solution is to exclude and dismiss certain methods without any benefit. Excluded methods of image manipulation

How-to read Results of chosen Procedures: As mentioned: Benchmark is a commercial package and is defined as “100%”. 700 documents, 1’400 pages, according to full text analysis 704 upto max. 709 SwissQR in them. Processing time and hit rate of every solution / (additional) fallback step is measuread and analyzed. Absolute processing time is not relevant here as our use case, I think, is more complex. Maybe the delta can be relevant to some of you.

Results of the Benchmark:

  1. Benchmark: Commercial package recognized 638 (out of actually 704+ SwissQR Codes): defined as 100%
  2. ZXing.Net: Run “default”: Full page, no image manipulation, solely set “TRY_HARDER” Recognized 536 (out of actually 704+ SwissQR Codes: 84.0% of benchmark)
  3. Final solution (details see below) with a combination of methods PLUS a second QR Decoder: Recognized 703 (out of actually 704+ SwissQR Codes: 110.2% of benchmark)
Chris21x commented 7 months ago

Post 3 / 3

Our Solution Approach:

Hint: We’re talking (partly) of image processing, make sure your memory processing, maintenance and clearing covers all cases!

Our actual resulting Solution: The order of the steps focus on “max. additional hit rates” considering also the overall processing time.

  1. Read full page with ZXING without image processing with the switch “TRY_HARDER” Result here: 583 out of 704+ (84.0% of benchmark)
  2. Read full page with “QRCodeDecoderLibrary” Result here: additional 97 SwissQR codes were found (adding +15.2% of benchmark)
  3. Read CROPPED bottom part of the page (acc. to specification) with ZXING, shrink image by 50% with the switch “TRY_HARDER” Result here: additional 20 SwissQR codes were found (adding +2.9% of benchmark)
  4. Read full page with “QRCodeDecoderLibrary”, shrink image by 50% Result here: additional 27 SwissQR codes were found (adding +4.2% of benchmark)
  5. Read CROPPED bottom part of the page (acc. to specification) with ZXING, enlarge image to 200% with the switch “TRY_HARDER” Result here: additional 12 SwissQR codes were found (adding +1.9% of benchmark)
  6. Read full page with ZXING, shrink image by 50% with the switch “TRY_HARDER” Result here: additional 11 SwissQR codes were found (adding +1.7% of benchmark)

Questions are welcome. I hope I could help other coding friends who face such a challenge. Personal comment: The SwissQR is crap by definition. A selected medium redundancy (15%) built in and already occupying half of this by the “Swiss Cross” and its border in the middle: this is near to “bound to fail” in the real life in the first place.

Kind regards from Switzerland - Chris

axxel commented 7 months ago

@Chris21x Thanks for the detailed explanation of your approach and your results. To lure you into giving ZXingCpp a try, I will now boldly claim that ZXingCpp with a single pass will surpass your 110% results and do that 2 to 10x faster that what you currently use. I'll happily accept defeat if that should not be true ;).

Chris21x commented 7 months ago

@axxel Fair enough - Challenge accepted. I do LOVE such things. Especially as the goal is to potentially give everyone around additional know-how and information. Last but not least: additional information to you, the developer. I thought about a reasonable approach:

  1. To give you a fair shot, we need to get a broader document set / base of examples (Remember? 703 out of maybe 704, maybe only a few more, SwissQR were read) - it could also be, that there is only ONE additional in the documents and that one really is corrupt - then you would have no chance to superceed. I'm thinking of 2'000+ documents. I do have another sample set of 17k documents for my OCR stress tests but these are unclassified and the most part is no invoice...)
  2. In order not to create too much work here to adopt the whole of our Client/Server Solution (surely would take some days), I would run on this expanded & different (!) document basis:
    • The processing with our package using the commercial solution (defining this as 100% "benchmark")
    • The processing with our above described solution
    • The processing with a dedicated piece of software using basically your (expanded) example on your GitHub page

The result would be a pure comparison of the recognition rates. Therefore in this first test batch we can't compare processing time/speed (But IF you're successful, in a following step we would check this in a next dedicated test).

What do you think?

What I do want to ask you: to be able to create a real comparison, can you adjust your example code (which reads all kinds of barcodes) to a dedicated one for SwissQR incl. switches like TRY_HARDER? I can do this as well but it would take more time. I'm a bit short of time currently...

Conditions for counter-challenge approved?

:-D

PS: Your source code framework for this challenge should be a simple cmd accepting the file name including path of a single PDF. Ideally: Create a ZIP Package including a VS Project.

axxel commented 7 months ago

Using the sample set you already have is perfectly fine with me. I did not realize that those 110% are basically 100% of all detectable symbols. Using a different one that poses the same challenges as in the OP is also fine. As for limiting the search space, the sample code already support that, you simply have to pass "QRCode" as the last parameter ('barcode-format-list'). tryHarder is enabled by default and should stay on.

ZXingCpp does not operate on PDFs, only on pixel data. But so does ZXing.NET if I'm not mistaken. I have no idea how to canonically render a PDF into a bitmap using .NET infrastructure. If you need help with how to construct an ImageView object from your pixel data, let me know what kind of image/buffer data structure you are working with.

Chris21x commented 7 months ago

That's fine, I will use the same PDF renderer and settings as we do in our package. This will now take some time, as I said, I'm currently a bit short on time.

Chris21x commented 7 months ago

@axxel - Our Challenge Just as interim information: I chose and prepared the testing document basis: the test document set is kinda "worst case":

Will proceed as follows, starting this evening:

  1. Run these say 6.4k documents through the commercial package (takes roughly 12+ hrs) & analyze This result is used as benchmark = 100%
  2. Run these documents through our solution described above & analyze
  3. Run these documents through a "mock-up" with your alpha-Package & analyze

Will take surely a week or two - has to run during the night(s). Processing time will not be analyzed - not comparable. (Would be next step).

Kind greetz from Switzerland - Chris

PS: Is here any way to send a direct message to you?

axxel commented 7 months ago

@Chris21x thanks for the update. You can reach me under zxingcpp at gmail dot com.

husam-ebish commented 2 months ago

Hello everyone, This issue has been open since February 2022. Is there no solution yet? I would appreciate it if someone could provide sample code for the solution.

Thank you!

Chris21x commented 2 months ago

Dear @husam-ebish (and dear @axxel)

Thank you for your question. At the moment I didn't follow up on this topic, as I simply CAN'T introduce any .net Core into my project - it's in some core libraries incompatible. Would it be important and helpfull for you? Then I'll rethink. I implemented above described procedure which ist "good enough" for our customers (but - to be honest - not for me as a developer) - see: https://github.com/micjahn/ZXing.Net/issues/408#issuecomment-2029651489

Please let me know, Husam. "The more people asking, the more likely...", blah-blah. I think, you know what I mean.

Sorry, @axxel for having given up for the time beeing. If it's incompatible to used libraries, it creates a huge amount of work.

Kind regards from Switzerland - Chris

husam-ebish commented 2 months ago

Hi @Chris21x

Thank you for your reply.

Yes, it is crucial to have a solution capable of reading the QR code on Swiss bills.

The implementation you mentioned is suitable for personal use or software that doesn't require ISO certification. However, this solution does not meet Swiss software quality standards, and therefore, it won't be accepted as part of Swiss certified software.

It would be ideal if we could provide an open-source solution that is supported by the Swiss community and Swiss companies.

Grüsse aus Züri