Performance of 1D Codes like EAN_13 and Code_39

[Antidote00]

Hello,

first of all: thanks for your great work. It was very easy to integrate the decoder into an existing QML project.

However, I observed some performance problems especially with 1D Barcodes EAN_13 and Code_39 on both Android and Desktop. If I enable these two with QR, I have 70 FPS on Desktop Fedora (Thinkpad T490s) and 1 FPS on Android 9 (Moto x4). In comparison enabling QR_Code solely, the FPS on Desktop increase to 400 and 40 for Smartphone respectively. But as you could guess, I am more interested in running qzxing on a mobile phone supporting both 1D Codes and 2D codes simultaneously.

Now my question: Is this normal? Or even on purpose? Could this be a bug within my implementation? Does anyone of you know some performance boosts? Did I miss something? Unfortunately I couldn't find anything in the documentation. I have not yet studied the sources in depth, however, before that I wanted to ask here for experiences and advice.

Thanks in advance. br

Edit: I should add, that the slow performance of 1D Codes could be reproduced on the same setup with the example QZXINGLive of this repository. At the first start of the example, i received a warning "Detected problems with API compatibilty (visit ...).

[ftylitak]

Hello @Antidote00

interesting observation. Hopefully during the weekend i will do some tests to see if there is any quick win.

Apart from that, make sure that in the QML, you have set the tryHarder to false in QZXingFilter -> decoder -> tryHarder

QZXingFilter 
{
//...
  decoder {
    tryHarder: false
  }
///..
}

[Antidote00]

Hi @ftylitak

thanks for the response :). I disabled/enabled tryHarder several times. On Desktop tryHarder: false increases the FPS between 5 and 10. On Smartphone the FPS of the Scanner is still between 1 and 2. Note: The Videostream (VideoOutput) is of course perfect. No freezing and lags.

As is understood, tryHarder starts to rotate the image in 90 degree steps? I guess this is extremely useful for 1D Codes, especially as most people use their phones in portrait mode.

I tried some fairytale compiler opts, no success

QMAKE_CXXFLAGS_RELEASE -= -O2
QMAKE_CXXFLAGS_RELEASE += -O3
QMAKE_LFLAGS_RELEASE -= -O1

I will also investigate further and looking forward to your response :).

Edit:

Testing additional devices with Clang QT 5.12.5, Release Build -O3

Google Pixel 1 (Android 10 - 64 Bit) - 1 FPS Motorola Nexus 6 (Android 7.1 - 32 Bit) - 3 FPS Motorola x4 (Android 9 - 64 Bit) - 1 FPS Samsung Galaxy 4 Mini (Android 5.0 - 32Bit) - 3 FPS (Some Problems with Autofocus, however oldest but best working phone...)

The FPS were measured using the example application with:

...
        onDecodingFinished:
        {
           timePerFrameDecode = (decodeTime + framesDecoded * timePerFrameDecode) / (framesDecoded + 1);
           framesDecoded++;
           console.log("frame finished: " + succeeded, decodeTime, timePerFrameDecode, framesDecoded);
        }
...

While testing, the smartphone camera was not looking on a barcode, just randomly around the office. I guess this calls some kind of standard "let us look for some code" routine. Is this (1D) routine somehow limited by a timer? If a barcode or QR is then focused, you can observe some additional calculations, especially when tryHarder is true, so the decodeTime increases further. If a code was successfully recognized, the decodeTime decreases extremely, resulting in a very high frame rate.

I hope this information helps somewhat. I will now dive deeper into the code. Please let me know, if you require any data or additional tests.

[ftylitak]

Hello @Antidote00

I would suggest you try the change that I pushed in branch https://github.com/ftylitak/qzxing/tree/decoder-1d-performance-invest .

Practically, I refactored the decoding operation to minimize the decoding operations if tryHarder is not set.

Could you give it try and see if it works for you?

[ftylitak]

@Antidote00, by the way, thank you very much for your thorought testing!

[Antidote00]

As expected your fix highly increases the performance to ~10 fps. Making the mobile experience a lot better. Of course, this comes with the costs of a "lower" recognition rate, since we lose the rotation of the image as well as the benefits from hints.setTryHarder(true);.

However, the main question is still: why does e.g. EAN_13 performs so much slower then a QR_Code? Using only EAN_13 compared to using only QR results in a difference of 30 fps on mobile. The EAN_13 reader seems pretty straight forward to me, however all those functions calls between qzxing.cpp and the actual EAN_13 reader could of course "do the rest", haha.

Of course hints.setTryHarder(true); has some very costly tweaks, for example

...
  int maxLines;
  if (tryHarder) {
    maxLines = height; // Look at the whole image, not just the center
  } else {
    maxLines = 15; // 15 rows spaced 1/32 apart is roughly the middle half of the image
  }
...

in OneDReader.cpp. But .. mh.. I mean we have very powerful mobile devices, why could such an operation be that "slow"? In comparison to the ZXing Android App, I have the feeling we lack something here. Could it be my setup? To old / new Android NDK Version? (v20.0.05..).

Thanks a lot @ftylitak for this. It helped me a lot to understand what is going on :). I will further investigate. Do you still have some other performance tips?

[ftylitak]

Indeed our smartphones nowadays are pretty powerfull and I would expect high FPS. I wil try to look on it a bit further.

Till then, here is one further change that I would expect to have impact...though I have not seen much of a difference :P . Does it make any change (good or bad) to your project?

[Antidote00]

Actually it does changes something, but i am not sure if this is on purpose and i cant find out why this is happening. In OneDReader.cpp line 43 bool tryHarder = hints.getTryHarder();is always false. tryharder is true in qml and also in QZXing.cpp in line 433. Does something overwrite the boolean? This of course prevents finding barcodes that are rotated counter-clockwise.

Edit: Okay I guess i have found the error source. To many different exceptions are thrown now. You have to change line 52 and 55 in QRCodeReader.cpp to throw NotFoundException();

Edit2: With all the above made changes and both tryHarder set to true, I got ~ 4 FPS. What is waaay more satisfying then 1 FPS, haha. The Scanning works pretty good now, even with (in my case) very small (~ 3 cm) QR and Barcodes. Nevertheless I will further search for performance boosts on mobile devices as this will increase the user experience. And of course, adding additional barcode types will decrease the performance again.

Edit3: I am not sure what happened... i made some UI changes in QML, nothing specific.. now the FPS is again at 1... No major code changes. I will investigate this...

Edit4: Okay. I found what is most expensive in 1D scanning with tryHard true

...
  int rowStep = std::max(1, height >> (tryHarder ? 8 : 5));
  using namespace std;
  // cerr << "rS " << rowStep << " " << height << " " << tryHarder << endl;
  int maxLines;
  if (tryHarder) {
    maxLines = height; // Look at the whole image, not just the center. NOTE: This is EXTREMELY costly on mobile...
  } else {
    maxLines = 15; // 15 rows spaced 1/32 apart is roughly the middle half of the image
  }
...

in OneDReader.h.

I am also curious why we are only finding Barcodes in Grayscale images. The XZing Android applications uses YUV color format. Do they make use of the chromatic components UV?

[axxel]

I stumbled upon this issue and could not resist to use this opportunity to advertise the 'new c++' port of zxing because I spent months of work on the performance and accuracy improvement of those algorithms and especially the 1D readers profited with a 5-10x speedup. A tryHarder == true scan of a 1280x780 image with all formats enabled (but no barcode visible) takes about 20ms on my Pixel 3 with our android wrapper, while the upstream Java code takes 180ms.

I tried to get a quick performance comparison between QZXingLive and our equivalent ZXingQtCamReader by forcing QZXingLive to operate on the whole 640x480 image of my Dell XPS webcam in tryHarder mode and saw a 5x speed advantage of 'our' implementation.

The possibility of replacing the zxing-cpp backend of QZXing with the nu-book/zxing-cpp implementation has been discussed here already 2 years ago and @ftylitak decided to not do that at the time due to concerns about requiring a C++-14 tool chain. As of today, we actually depend on a C++-17 compiler but so does Qt 6 (just saying ;)).