QZXing for Qt 6.2

[dabbinavo]

As mentioned in the Qt Development Mailing List, the first shot of implementation of qtmultimedia for Qt 6 is finished and merged back into the dev branch of the module.

Are there plans of QZXing supporting Qt 6.2? If so, when is development for it going to be started? Qt 6.2 feature freeze, first beta or release of Qt 6.2.0?

If i understand the Readme correctly, the issue with the missing QTextCodec Module is already solved by using QStringDecoder. The missing QML modules (Camera and Video) may now be present again in the dev branch of qtmultimedia.

I think there are many QZXing users like me, that are already waiting for the Qt 6.2 release - hopefully including all missing functionalities of Qt 5.15 again.

[ftylitak]

Hello @dabbinavo

thank you very much for the above information.

Regarding the QTextCodec, QStringDecoder is already in use when compiling for Qt 6. Though keep in mind that it still does not support as many encodings as QTextCodec used to support. But this is a compromise that we have to accept for now. Here is a screenshot comparison between the supported codecs:

Regarding the support of Qt Multimedia in Qt 6, I believe the safest would be to support it upon Qt 6.2 beta release. Based on the Qt 6.2 release wiki, the first beta will be available in July 2021 so expect by the middle-to-end of July to be also supported by QZXing.

(this issue will be kept open for reference till then)

[ftylitak]

A status update for Qt 6.2 Multimedia support, a week ago I have made a try to gather info on how to approach the new solution though even for Qt 6.2 Beta there is no complete documentation update yet.

Thus, its support will be postponed for September (August vacations :P) hopping that it will require less guessing on how to solve it.

A "useful" information that provided a good direction was extracted from the Qt development mailing list that states:

• Removed QAbstractVideoFilter. For filtering QML video, use shader effects instead.

[permotion88]

@ftylitak Qt 6.2.0 was released on 30.09.2021, did you manage to find some time to do some experiments to adjust QZXing for Qt 6? Regards and thanks for your work so far.

[kiibimees]

In Qt 6, it should be possible to use QVideoSink like this:

#include <QObject>
#include <QDebug>
#include <QtConcurrent>
#include <QtMultimedia/QVideoSink>
#include <QtMultimedia/QVideoFrame>
#include <QtQml/qqmlregistration.h>
#include <QZXing>

class CodeDecoderVideoSink : public QObject {
    Q_OBJECT
    QML_ELEMENT
    Q_PROPERTY(QObject* videoSink WRITE setVideoSink)
public:
    explicit CodeDecoderVideoSink(QObject *parent = nullptr){
        decoder.setDecoder(QZXing::DecoderFormat_QR_CODE);
        decoder.setSourceFilterType(QZXing::SourceFilter_ImageNormal);
        decoder.setTryHarderBehaviour(QZXing::TryHarderBehaviour_ThoroughScanning | QZXing::TryHarderBehaviour_Rotate);

        connect(&decoder, &QZXing::tagFound, this, &CodeDecoderVideoSink::tagFound);
    }

    void setVideoSink(QObject *videoSink){
        m_videoSink = qobject_cast<QVideoSink*>(videoSink);

        connect(m_videoSink, &QVideoSink::videoFrameChanged, this, [=](const QVideoFrame &frame){
            static bool m_decoding = false;
            if(!m_decoding){
                m_decoding = true;
                auto image = frame.toImage();//.convertToFormat(QImage::Format_Grayscale8);
                QtConcurrent::run([=](){
                    decoder.decodeImage(image, image.width(), image.height());
                    m_decoding = false;
                });
            }
        });
    }

signals:
    void tagFound(QString tag);

private:
    QZXing decoder;
    QVideoSink *m_videoSink;
};

QML:

CaptureSession {
    camera: Camera {
        active: true
    }
    videoOutput: videoOutput
}
VideoOutput{
    id: videoOutput
}

CodeDecoderVideoFilter{
    videoSink: videoOutput.videoSink
    onTagFound: {
        console.log("Tag found!!!!!")
    }
}

Everything seems to work. The only trouble is that it doesn't recognize any codes :D I'm not sure if it's about image format or something else.

[ftylitak]

@kiibimees thank you very much for valuable input. I have tested your code on Ubuntu 21.10 and it worked perfectly!

Here are the files of the sample application that I have used for testing. Within the coming days I will integrate it into the core part of QZXing. Thank you very much once again!

VideoSinkQZXing.pro

CONFIG += qzxing_multimedia \
          enable_decoder_1d_barcodes \
          enable_decoder_qr_code \
          enable_decoder_data_matrix \
          enable_decoder_aztec \
          enable_decoder_pdf17

include(../../qzxing/src/QZXing-components.pri)

QT += quick
QT += multimedia
QT += concurrent

CONFIG += c++11
SOURCES += \
        main.cpp

RESOURCES += qml.qrc

# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

HEADERS += \
    codedecodervideosink.h

main.cpp

#include <QGuiApplication>
#include <QQmlApplicationEngine>
#include <QQmlContext>
#include "codedecodervideosink.h"

#include "QZXing.h"

int main(int argc, char *argv[])
{
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
    QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
#endif

    QGuiApplication app(argc, argv);
     QZXing::registerQMLTypes();

    QQmlApplicationEngine engine;
    const QUrl url(QStringLiteral("qrc:/main.qml"));
      qmlRegisterType<CodeDecoderVideoSink>("CodeDecoderVideoSink", 1, 0, "CodeDecoderVideoSink");
    QObject::connect(&engine, &QQmlApplicationEngine::objectCreated,
                     &app, [url](QObject *obj, const QUrl &objUrl) {
        if (!obj && url == objUrl)
            QCoreApplication::exit(-1);
    }, Qt::QueuedConnection);
    engine.load(url);

    return app.exec();
}

codedecodervideosink.h

#ifndef CODEDECODERVIDEOSINK_H
#define CODEDECODERVIDEOSINK_H

#include <QObject>
#include <QDebug>
#include <QtConcurrent>
#include <QtMultimedia/QVideoSink>
#include <QtMultimedia/QVideoFrame>
#include <QtQml/qqmlregistration.h>
#include "QZXing.h"

class CodeDecoderVideoSink : public QObject {
    Q_OBJECT
    QML_ELEMENT
    Q_PROPERTY(QObject* videoSink WRITE setVideoSink)
public:
    explicit CodeDecoderVideoSink(QObject *parent = nullptr){
        decoder.setDecoder(QZXing::DecoderFormat_QR_CODE);
        decoder.setSourceFilterType(QZXing::SourceFilter_ImageNormal);
        decoder.setTryHarderBehaviour(QZXing::TryHarderBehaviour_ThoroughScanning | QZXing::TryHarderBehaviour_Rotate);

        connect(&decoder, &QZXing::tagFound, this, &CodeDecoderVideoSink::tagFound);
    }

    void setVideoSink(QObject *videoSink){
        m_videoSink = qobject_cast<QVideoSink*>(videoSink);

        connect(m_videoSink, &QVideoSink::videoFrameChanged, this, [=](const QVideoFrame &frame){
            static bool m_decoding = false;
            if(!m_decoding){
                m_decoding = true;
                auto image = frame.toImage();//.convertToFormat(QImage::Format_Grayscale8);
                QtConcurrent::run([=](){
                    decoder.decodeImage(image, image.width(), image.height());
                    m_decoding = false;
                });
            }
        });
    }

signals:
    void tagFound(QString tag);

private:
    QZXing decoder;
    QVideoSink *m_videoSink;
};

#endif // CODEDECODERVIDEOSINK_H

main.qml

import QtQuick
import QtQuick.Window
import QtQuick.Controls
import QtQuick.Layouts
import QtMultimedia
import CodeDecoderVideoSink 1.0

ApplicationWindow
{
    id: window
    visible: true
    width: 640
    height: 480
    title: "Qt QZXing Filter Test"

    CaptureSession {
        camera: Camera {
            active: true
        }
        videoOutput: videoOutput
    }
    VideoOutput{
        id: videoOutput
    }

    CodeDecoderVideoSink{
        videoSink: videoOutput.videoSink
        onTagFound: {
            console.log("Tag found!!!!!", tag)
        }
    }
}

[kiibimees]

I'm glad to hear that my proposal was helpful. I tried also with nu-book/zxing-cpp to see if I can detect any tags and I was able to get it working. I guess I have to try QZXing once again to see if it's about my video format (Windows+WebCam) or was it something else. Goot to know the confirmation that it works on Linux.

NB! Unfortunately this method doesn't work on Android. I reported a bug abut it: https://bugreports.qt.io/browse/QTBUG-97789

[kiibimees]

@ftylitak , you can read my comment on Qt bugreports to see how to make it work(around:) on Android.

[ftylitak]

Greetings,

a new branch has been created (qt6_2_multimedia_support ) that includes the support of Qt 6.2 multimedia.

QZXingFilterVideoSink class holds the implementation of QZXingFilter using QVideoSink. Also QZXingLive example has been updated, https://github.com/ftylitak/qzxing/tree/qt6_2_multimedia_support/examples/QZXingLive/main_qt6_2.qml file shows an example on how to use it.

QZXingFilter API has remained the same except of 2 things:

• videoSink property must be set using VideoOutput.videoSink
• the capureRect (if used) is calculated through a different procedure

    Camera
    {
        id:camera
        active: true
        focusMode: Camera.FocusModeAutoNear
    }

    CaptureSession {
        camera: camera
        videoOutput: videoOutput
    }

    VideoOutput
    {
        id: videoOutput
        anchors.top: text1.bottom
        anchors.bottom: text2.top
        anchors.left: parent.left
        anchors.right: parent.right
        fillMode: VideoOutput.Stretch

        property double captureRectStartFactorX: 0.25
        property double captureRectStartFactorY: 0.25
        property double captureRectFactorWidth: 0.5
        property double captureRectFactorHeight: 0.5

        Rectangle {
            id: captureZone
            color: "red"
            opacity: 0.2
            width: parent.width * parent.captureRectFactorWidth
            height: parent.height * parent.captureRectFactorHeight
            x: parent.width * parent.captureRectStartFactorX
            y: parent.height * parent.captureRectStartFactorY
        }
    }

    QZXingFilter
    {
        id: zxingFilter
        videoSink: videoOutput.videoSink

        captureRect: {
            videoOutput.sourceRect;
            return Qt.rect(videoOutput.sourceRect.width * videoOutput.captureRectStartFactorX,
                           videoOutput.sourceRect.height * videoOutput.captureRectStartFactorY,
                           videoOutput.sourceRect.width * videoOutput.captureRectFactorWidth,
                           videoOutput.sourceRect.height * videoOutput.captureRectFactorHeight)
        }

        decoder {
            enabledDecoders: QZXing.DecoderFormat_EAN_13 | QZXing.DecoderFormat_CODE_39 | QZXing.DecoderFormat_QR_CODE

            onTagFound: {
                console.log(tag + " | " + decoder.foundedFormat() + " | " + decoder.charSet());

                window.detectedTags++;
                window.lastTag = tag;
            }

            tryHarder: false
        }

        onDecodingStarted:
        {
//            console.log("started");
        }

        property int framesDecoded: 0
        property real timePerFrameDecode: 0

        onDecodingFinished:
        {
           timePerFrameDecode = (decodeTime + framesDecoded * timePerFrameDecode) / (framesDecoded + 1);
           framesDecoded++;
           if(succeeded)
            console.log("frame finished: " + succeeded, decodeTime, timePerFrameDecode, framesDecoded);
        }
    }

The implementation is considered completed though and it will be merged to master after further validations and testing.

[deletexl]

Is this code working? I'm using Qt 6.2.1, I have a code for android, but the camera does not open, I encounter a white screen. Did your example work?

My Code

import QtQuick
import QtQuick.Controls
import QtMultimedia

ApplicationWindow {
    id: rootWindow
    visible: true
    width: 375 * dp
    height: 812 * dp

    MediaDevices {
        id: deviceList
    }

    Camera {
        id: mainCamera
        active: true
        cameraDevice: deviceList.defaultVideoInput
        focusMode: Camera.FocusModeAutoNear
    }

    CaptureSession {
        camera: mainCamera
        videoOutput: videoOutput
    }

    VideoOutput {
        id: videoOutput
        anchors.fill: parent
        fillMode: VideoOutput.PreserveAspectFit
    }
}

Code output:

I Camera  : open camera: 0, package name: com.package.camera
D HwFrameworkSecurityPartsFactory: HwFrameworkSecurityPartsFactory in.
I HwFrameworkSecurityPartsFactory: add HwFrameworkSecurityPartsFactory to memory.
I HwCameraUtil: notifySurfaceFlingerCameraStatus : isFront = false , isOpend = true
I Camera  : close camera: 0, package name: com.package.camera
I HwCameraUtil: notifySurfaceFlingerCameraStatus : isFront = false , isOpend = false
I Camera  : open camera: 1, package name: com.package.camera
I HwCameraUtil: notifySurfaceFlingerCameraStatus : isFront = true , isOpend = true
I HwCameraUtil: notifySurfaceFlingerFrontCameraStatus 8011 transact success!
I Camera  : close camera: 1, package name: com.package.camera
I HwCameraUtil: notifySurfaceFlingerCameraStatus : isFront = true , isOpend = false
I HwCameraUtil: notifySurfaceFlingerFrontCameraStatus 8012 transact success!

[ftylitak]

@deletexl even though your application is not 100% related to this library, still it is an issue that I faced as well.

The issue is that the camera is trying to be activated probably at a wrong time during application startup. Not sure why it does not start though it is sure that you are seeing white screen becase the Camera is not active.

Try the following "silly" workaround to see if this works (again, I am not yet sure which approach is the optimal for this issue). It uses a Timer that will re-request the camera activation.

import QtQuick
import QtQuick.Controls
import QtMultimedia

ApplicationWindow {
    id: rootWindow
    visible: true
    width: 375 * dp
    height: 812 * dp

    MediaDevices {
        id: deviceList
    }

    Camera {
        id: mainCamera
        active: true
        cameraDevice: deviceList.defaultVideoInput
        focusMode: Camera.FocusModeAutoNear
    }

    CaptureSession {
        camera: mainCamera
        videoOutput: videoOutput
    }

    VideoOutput {
        id: videoOutput
        anchors.fill: parent
        fillMode: VideoOutput.PreserveAspectFit
    }

    Timer {
        interval: 500; running: true; repeat: false
        onTriggered: mainCamera.active = true
    }
}

[deletexl]

Camera
    {
        id:camera
        active: true
        focusMode: Camera.FocusModeAutoNear
    }

    CaptureSession {
        camera: camera
        videoOutput: videoOutput
    }

    VideoOutput
    {
        id: videoOutput
        anchors.top: text1.bottom
        anchors.bottom: text2.top
        anchors.left: parent.left
        anchors.right: parent.right
        fillMode: VideoOutput.Stretch

        property double captureRectStartFactorX: 0.25
        property double captureRectStartFactorY: 0.25
        property double captureRectFactorWidth: 0.5
        property double captureRectFactorHeight: 0.5

        Rectangle {
            id: captureZone
            color: "red"
            opacity: 0.2
            width: parent.width * parent.captureRectFactorWidth
            height: parent.height * parent.captureRectFactorHeight
            x: parent.width * parent.captureRectStartFactorX
            y: parent.height * parent.captureRectStartFactorY
        }
    }

    QZXingFilter
    {
        id: zxingFilter
        videoSink: videoOutput.videoSink

        captureRect: {
            videoOutput.sourceRect;
            return Qt.rect(videoOutput.sourceRect.width * videoOutput.captureRectStartFactorX,
                           videoOutput.sourceRect.height * videoOutput.captureRectStartFactorY,
                           videoOutput.sourceRect.width * videoOutput.captureRectFactorWidth,
                           videoOutput.sourceRect.height * videoOutput.captureRectFactorHeight)
        }

        decoder {
            enabledDecoders: QZXing.DecoderFormat_EAN_13 | QZXing.DecoderFormat_CODE_39 | QZXing.DecoderFormat_QR_CODE

            onTagFound: {
                console.log(tag + " | " + decoder.foundedFormat() + " | " + decoder.charSet());

                window.detectedTags++;
                window.lastTag = tag;
            }

            tryHarder: false
        }

        onDecodingStarted:
        {
//            console.log("started");
        }

        property int framesDecoded: 0
        property real timePerFrameDecode: 0

        onDecodingFinished:
        {
           timePerFrameDecode = (decodeTime + framesDecoded * timePerFrameDecode) / (framesDecoded + 1);
           framesDecoded++;
           if(succeeded)
            console.log("frame finished: " + succeeded, decodeTime, timePerFrameDecode, framesDecoded);
        }
    }

Yes, the situation I mentioned was a different problem, we solved it, you can find it here.

https://bugreports.qt.io/browse/QTBUG-98559

But I ran into a different problem and when I ran the example you gave, I got this error. I leave the output below

W libQRScannerarm64-v8a.so: QImage QVideoFrame::toImage() const : unsupported pixel format "Format""SamplerExternalOES" D libQRScanner_arm64-v8a.so: QZXingFilter error: Cant create image file to process. W OpenGLRenderer: [SurfaceTexture-0-16181-0] bindTextureImage: clearing GL error: 0x500 E OpenGLRenderer: [SurfaceTexture-0-16181-0] checkAndUpdateEglState: invalid current EGLContext W System.err: java.lang.IllegalStateException: Unable to update texture contents (see logcat for details) W System.err: at android.graphics.SurfaceTexture.nativeUpdateTexImage(Native Method) W System.err: at android.graphics.SurfaceTexture.updateTexImage(SurfaceTexture.java:248) F libQRScanner_arm64-v8a.so: Cannot make QOpenGLContext current in a different thread F libc : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 16435 (QtThread), pid 16181

[ftylitak]

@deletexl thank you for referencing the bug report!

The crash that you were experiencing was caused by the multithreaded approach that is not supported for Android in the way that it is implemented at least.

The downside of the thread-less implementation is the slugginess of the video feed which I hope will be fixed soon.

[deletexl]

@ftylitak I did apply your recent changes, my application did not crash. But as you mentioned, there is a slowdown. What to do to fix this? How may i help you ?

[ftylitak]

@deletexl

I have tries some tunings from the previous implementation of QZXingFilter that made some minor improvements. Though the big issue here is that we are unable to do the processing in a different thread to allow the video feed to be smooth. I will suppose that this is a bug of the Qt Multimedia.

I am currently trying to find a way for the multithreading to work and not cause SIGABRT.

[kiibimees]

@ftylitak , It IS possible to use multithreaded approach. You have missed the detail that map/unmap must be called for EVERY FRAME. Regardless if you use it or not. Have a look at my comment and example code: https://bugreports.qt.io/browse/QTBUG-97789?focusedCommentId=592553&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-592553

[ftylitak]

@kiibimees indeed you are right. By mistake I though it was already following your example. Now it should be ok without that much of a lag in the video feed.

[deletexl]

I test in Android 10 with QT 6.2 QML. It's work like charm !

[ApiTracer]

I have made a minimum working example for qt6 with cmake example of qzxing live. I can send you the project if you are interested. However, detection perfs is quite bad and video output freeze at regular interval.

[EndrII]

Any news when do your planed merge qt 6.2 support into main (master) branch?

[ftylitak]

All changes have been merged and version v3.3 has just been released with the support of Qt 6.2. This issue is considered closed. Any problem that might came up, should be opened as a new issue.