opencv / opencv_contrib

Repository for OpenCV's extra modules
Apache License 2.0
9.31k stars 5.74k forks source link

Does WechatQR output the four corners of the code? #2884

Open JianWang-CMU opened 3 years ago

JianWang-CMU commented 3 years ago

WechatQR could output the four corners of the bounding box of the QR code (like the red stars in the figure below). Can it output the four corners of the QR code itself? Thanks untitled

dddzg commented 3 years ago

Yes, in this version, we return the output from the CNN detector (just like your visualization figure). During the decoding, we will find the explicit corners. But it is not returned to the external interface yet. PR is welcomed. And we will support it in the future version if necessary.

JianWang-CMU commented 3 years ago

I read the WechatQR code to some degree. It uses a new NN-based detector to find the code and crop the code region. And then use a super-resolution neural network to enhance the image. Later, it calls ZXing to find the four corners (does WechatQR has an improvement on finding the four corners?) and decode the image, right? Thanks!

dddzg commented 3 years ago

Your understanding of the pipeline is right! But our zxing module is different from the original zxing module. We perform a lot of improvements in zxing module.

Specifically, besides traditional horizontal line searching (1:1:3:1:1 pattern), we propose an area size-based finder pattern detection method. we calculate the area size of the black and white block (25:16:9 pattern) to locate the finder pattern by the pre-computed connected cells.

JianWang-CMU commented 3 years ago

Great! When I used WechatQR without DNN, it also can do a better job than ZXing. Is this because of the improved corner detection method? Is there a document about the corner detection method? I feel a bit hard to read the code without any document/

dddzg commented 3 years ago

Yes, without DNN, our module is still better than the original zxing. I am so sorry that there is no document yet. I admit that it is hard to read our zxing module. Maybe you can start from https://github.com/opencv/opencv_contrib/blob/0165d48da48b169420cd4d70867b991bd48543e8/modules/wechat_qrcode/src/zxing/qrcode/qrcode_reader.cpp#L58.

The top-left,top-right,bottom-left, 3 corners are named as pattern, so there are finder_pattern_finder.cpp. The bottom-right corner is named as alignment_pattern, so there are alignment_pattern_finder.cpp. And we are also trying to make it simpler. You can try to read and run the code with some input sample images.

Yolanda-Gao commented 3 years ago

Great to find this question! It seems WechatQR can decode small QR codes (even without DNN; with DNN is better). I just tried several QR codes where the sampling ratio is about 1.3 (1.3Nx1.3N camera pixels for an NxN QR code), and WechatQR works. I am wondering how small can the code be? Does the version and error correction level (and others like rotation angle) matter? Is there a report about it? Thanks!

dddzg commented 3 years ago

I am sorry that there is no public quantitive evaluation about what size/version/rotation angle qrcode be decoded. It is welcomed to evaluate our qrcode module with public qrcode datasets. We are also looking forward to a report.

SantiAlex commented 3 years ago

Yes, in this version, we return the output from the CNN detector (just like your visualization figure). During the decoding, we will find the explicit corners. But it is not returned to the external interface yet. PR is welcomed. And we will support it in the future version if necessary.

NECESSARY!!! A new interface returns qr_code corners is useful.