`emscripten_set_mousedown_callback` does not call the callback function from Qt Creator

8Observer8 commented 1 week ago

The next example works without problems when I compile and run it from the command line: emcc -g main.cpp -o public/index.html But when I create Empty qmake Project from Qt Creater and run it - it doesn't work - it does not print "Mouse down" to the console:

main.cpp

#include <emscripten.h>
#include <emscripten/html5.h>
#include <iostream>

// https://github.com/emscripten-core/emscripten/issues/19266
int main()
{
    // Mouse Button Down:
    emscripten_set_mousedown_callback(
        EMSCRIPTEN_EVENT_TARGET_WINDOW, nullptr, 0, +[](int eventType,
        const EmscriptenMouseEvent *e, void *userData) -> EM_BOOL
        {
            std::cout << "Mouse down: " << e->button << "\n";
            return EM_FALSE;
        });
}

I create a project in Qt Creator like this:

sbc100 commented 1 week ago

Most likely if you use Qt you probably want to let it handle all the interaction with he mouse etc. Trying to do you own mouse interaction underneath Qt will likely cause confusion or maybe just doesn't work?

8Observer8 commented 1 week ago

The contributor of Qt said in the comments: https://bugreports.qt.io/browse/QTBUG-126513 that:

You need to handle this in a javascript mouse event and not a Qt mouse event. Once it gets to Qt mouse handler, the original javascript event handler has passed.

I have tried to use it:

void mousePressEvent(QMouseEvent *event) override
{
    EMSCRIPTEN_RESULT result = emscripten_request_pointerlock(EMSCRIPTEN_EVENT_TARGET_DOCUMENT, false);
    qDebug() << result;
}

It returns EMSCRIPTEN_RESULT_NOT_SUPPORTED

sbc100 commented 1 week ago

I see.. they you need to handle the event directly so that you can request the pointer lock.

In that case yes you probably do need to register your own mousedown callback. I'm not sure why shouldn't able to have your one registered alongside the Qt one.. I think they should all fire and we don't use stopPropagation within emscripten so I don't see why you wouldn't get both event.

8Observer8 commented 4 days ago

How do you think, guys, is it a bug of Qt or Emscripten? https://bugreports.qt.io/browse/QTBUG-126582

sbc100 commented 4 days ago

My guess is that its an emscripten issue, but I'm not sure how this API is supposed to work TBH. Needs more investigation.

8Observer8 commented 3 days ago

The main goal is to get movementX and movementY values from this function:

EM_JS(void, requestPointerLock, (), {
    document.body.requestPointerLock();

    document.body.onmousemove = (event) => {
        console.log(event.movementX, event.movementY);
    };
})

And print them here:

    void mouseMoveEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);

        // qDebug() << movementX << movementY;
    }

sbc100 commented 3 days ago

Sorry where are the above code snippets from? is that code from inside of Qt or from your project?

I though that goal here was to be able to call emscripten_request_pointerlock?

8Observer8 commented 3 days ago

I try another ways to solve the problem. emscripten_request_pointerlock doesn't work with Qt but this code works:

EM_JS(void, requestPointerLock, (), {
    document.body.requestPointerLock();

    document.body.onmousemove = (event) => {
        console.log(event.movementX, event.movementY);
    };
})

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);
        requestPointerLock();
    }

It locks the mouse pointer and prints movementX and movementY to the console when I have the mouse pointer. But I don't know how to get movementX and movementY in C++ code when I move the mouse pointer.

sbc100 commented 3 days ago

I try another ways to solve the problem. emscripten_request_pointerlock doesn't work with Qt but this code works:
EM_JS(void, requestPointerLock, (), {
    document.body.requestPointerLock();

    document.body.onmousemove = (event) => {
        console.log(event.movementX, event.movementY);
    };
})
    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);
        requestPointerLock();
    }
It locks the mouse pointer and prints movementX and movementY to the console when I have the mouse pointer. But I don't know how to get movementX and movementY in C++ code when I move the mouse pointer.

I'm not sure that is the right solution.. but if you want to go that route that is certainly questions for Qt. You would need to look at how to create/injext new Qt events, mostly likely take a look at the Qt emscripten implemenation.

However, I a little confused because your requestPointerLock above is still not running inside an event handler.. how is it working in that case?

sbc100 commented 3 days ago

Oh wait.. did you try passing true as the second argument (deferUntilInEventHandler) to emscripten_request_pointerlock ?

8Observer8 commented 3 days ago

Oh wait.. did you try passing true as the second argument (deferUntilInEventHandler) to emscripten_request_pointerlock ?

I tried. The mouse pointer is not locked. It prints -1 to the console when I click. This is the WASM output: dist.zip

#include <QtGui/QMouseEvent>
#include <QtGui/QOpenGLFunctions>
#include <QtOpenGL/QOpenGLWindow>
#include <QtWidgets/QApplication>

#ifdef Q_OS_WASM
#include <emscripten.h>
#include <emscripten/html5.h>
#endif

class OpenGLWindow : public QOpenGLWindow, private QOpenGLFunctions
{

public:

    OpenGLWindow()
    {
        setTitle("OpenGL ES 2.0, Qt6, C++");
        resize(350, 350);
    }

private:

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock(
            EMSCRIPTEN_EVENT_TARGET_DOCUMENT, true);
        qDebug() << result;
    }

    virtual void initializeGL() override
    {
        initializeOpenGLFunctions();
        glClearColor(0.2f, 0.2f, 0.2f, 1.f);
    }

    virtual void paintGL() override
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
};

int main(int argc, char *argv[])
{
    QApplication::setAttribute(Qt::ApplicationAttribute::AA_UseDesktopOpenGL);
    QApplication app(argc, argv);
    OpenGLWindow w;
    w.show();
    return app.exec();
}

pro

QT += core gui openglwidgets

win32: LIBS += -lopengl32

CONFIG += c++17

wasm: INCLUDEPATH += "C:\emsdk\upstream\emscripten\cache\sysroot\include"

SOURCES += \
    main.cpp

8Observer8 commented 3 days ago

However, I a little confused because your requestPointerLock above is still not running inside an event handler.. how is it working in that case?

I don't know but the mouse cursor was locked and the movementX and movementY values was printed to the console:

#include <QtGui/QMouseEvent>
#include <QtGui/QOpenGLFunctions>
#include <QtOpenGL/QOpenGLWindow>
#include <QtWidgets/QApplication>

#ifdef Q_OS_WASM
#include <emscripten.h>
#include <emscripten/html5.h>
#endif

#ifdef Q_OS_WASM
EM_JS(void, requestPointerLock, (), {
    document.body.requestPointerLock();

    document.body.onmousemove = (event) => {
        console.log(event.movementX, event.movementY);
    };
})
#endif

class OpenGLWindow : public QOpenGLWindow, private QOpenGLFunctions
{

public:

    OpenGLWindow()
    {
        setTitle("OpenGL ES 2.0, Qt6, C++");
        resize(350, 350);
    }

private:

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);
        requestPointerLock();
    }

    virtual void initializeGL() override
    {
        initializeOpenGLFunctions();
        glClearColor(0.2f, 0.2f, 0.2f, 1.f);
    }

    virtual void paintGL() override
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
};

int main(int argc, char *argv[])
{
    QApplication::setAttribute(Qt::ApplicationAttribute::AA_UseDesktopOpenGL);
    QApplication app(argc, argv);
    OpenGLWindow w;
    w.show();
    return app.exec();
}

pro

QT += core gui openglwidgets

win32: LIBS += -lopengl32

CONFIG += c++17

wasm: INCLUDEPATH += "C:\emsdk\upstream\emscripten\cache\sysroot\include"

SOURCES += \
    main.cpp

sbc100 commented 3 days ago

Oh wait.. did you try passing true as the second argument (deferUntilInEventHandler) to emscripten_request_pointerlock ?

I tried. The mouse pointer is not locked. It prints -1 to the console when I click. This is the WASM output: dist.zip

#include <QtGui/QMouseEvent>
#include <QtGui/QOpenGLFunctions>
#include <QtOpenGL/QOpenGLWindow>
#include <QtWidgets/QApplication>

#ifdef Q_OS_WASM
#include <emscripten.h>
#include <emscripten/html5.h>
#endif

class OpenGLWindow : public QOpenGLWindow, private QOpenGLFunctions
{

public:

    OpenGLWindow()
    {
        setTitle("OpenGL ES 2.0, Qt6, C++");
        resize(350, 350);
    }

private:

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock(
            EMSCRIPTEN_EVENT_TARGET_DOCUMENT, true);
        qDebug() << result;
    }

    virtual void initializeGL() override
    {
        initializeOpenGLFunctions();
        glClearColor(0.2f, 0.2f, 0.2f, 1.f);
    }

    virtual void paintGL() override
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
};

int main(int argc, char *argv[])
{
    QApplication::setAttribute(Qt::ApplicationAttribute::AA_UseDesktopOpenGL);
    QApplication app(argc, argv);
    OpenGLWindow w;
    w.show();
    return app.exec();
}

pro

QT += core gui openglwidgets

win32: LIBS += -lopengl32

CONFIG += c++17

wasm: INCLUDEPATH += "C:\emsdk\upstream\emscripten\cache\sysroot\include"

SOURCES += \
    main.cpp

Can you step into the code for emscripten_request_pointerlock and see why its returning -1? It should be returning EMSCRIPTEN_RESULT_DEFERRED (1)

8Observer8 commented 3 days ago

Can you step into the code for emscripten_request_pointerlock and see why its returning -1? It should be returning EMSCRIPTEN_RESULT_DEFERRED (1)

It returns EMSCRIPTEN_RESULT_NOT_SUPPORTED (-1):

#define EMSCRIPTEN_RESULT_SUCCESS              0
#define EMSCRIPTEN_RESULT_DEFERRED             1
#define EMSCRIPTEN_RESULT_NOT_SUPPORTED       -1

I cannot set a breakpoint at the emscripten_request_pointerlock line because Qt Creator doesn't allow it:

But I can click on this link:

Result:

sbc100 commented 3 days ago

You should be able to set a breakpoint in devtools once you have your app running in chrome. You should be able to then step through the emscripten_request_pointerlock in devtools.

8Observer8 commented 3 days ago

A target is the document but target.requestPointerLock is undefined:

8Observer8 commented 2 days ago

I have reproduced the problem above in pure Emscripten from CMD: emcc -g main.cpp -o public/index.html

It returns -1 because target.requestPointerLock is undefined:

#include <emscripten.h>
#include <emscripten/html5.h>
#include <iostream>

int main()
{
    // Mouse Button Down:
    emscripten_set_mousedown_callback(
        "#canvas", nullptr, 0, +[](int eventType,
            const EmscriptenMouseEvent *e, void *userData) -> EM_BOOL
        {
            EMSCRIPTEN_RESULT result = emscripten_request_pointerlock(
                EMSCRIPTEN_EVENT_TARGET_DOCUMENT, true);
            std::cout << result << std::endl;
            return EM_FALSE;
        });

    return 0;
}

8Observer8 commented 2 days ago

document doesn't have requestPointerLock:

8Observer8 commented 2 days ago

I have replaced EMSCRIPTEN_EVENT_TARGET_DOCUMENT with #canvas and the example above works without the problem: emcc -g main.cpp -o public/index.html

main.cpp

#include <emscripten.h>
#include <emscripten/html5.h>
#include <iostream>

int main()
{
    // Mouse Button Down:
    emscripten_set_mousedown_callback(
        "#canvas", nullptr, 0, +[](int eventType,
            const EmscriptenMouseEvent *e, void *userData) -> EM_BOOL
        {
            EMSCRIPTEN_RESULT result = emscripten_request_pointerlock("#canvas", true);
            std::cout << result << std::endl;
            return EM_FALSE;
        });

    return 0;
}

But I cannot reproduce this solution for Qt because when I replace EMSCRIPTEN_EVENT_TARGET_DOCUMENT with #canvas the target is null:

It is because the Qt's canvas doesn't have id="canvas" for the <canvas> element:

8Observer8 commented 2 days ago

I have tried to replace EMSCRIPTEN_EVENT_TARGET_DOCUMENT with EMSCRIPTEN_EVENT_TARGET_SCREEN. I works for pure Emscripten: emcc -g main.cpp -o public/index.html

main.cpp

#include <emscripten.h>
#include <emscripten/html5.h>
#include <iostream>

int main()
{
    // Mouse Button Down:
    emscripten_set_mousedown_callback(
        "#canvas", nullptr, 0, +[](int eventType,
            const EmscriptenMouseEvent *e, void *userData) -> EM_BOOL
        {
            EMSCRIPTEN_RESULT result = emscripten_request_pointerlock(
                EMSCRIPTEN_EVENT_TARGET_SCREEN, true);
            std::cout << result << std::endl;
            return EM_FALSE;
        });

    return 0;
}

But why doesn't EMSCRIPTEN_EVENT_TARGET_SCREEN works in Qt?

#include <QtGui/QMouseEvent>
#include <QtGui/QOpenGLFunctions>
#include <QtOpenGL/QOpenGLWindow>
#include <QtWidgets/QApplication>

#ifdef Q_OS_WASM
#include <emscripten.h>
#include <emscripten/html5.h>
#endif

class OpenGLWindow : public QOpenGLWindow, private QOpenGLFunctions
{

public:

    OpenGLWindow()
    {
        setTitle("OpenGL ES 2.0, Qt6, C++");
        resize(350, 350);
    }

private:

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock(
            EMSCRIPTEN_EVENT_TARGET_SCREEN, true);
        qDebug() << result;
    }

    virtual void initializeGL() override
    {
        initializeOpenGLFunctions();
        glClearColor(0.2f, 0.2f, 0.2f, 1.f);
    }

    virtual void paintGL() override
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
};

int main(int argc, char *argv[])
{
    QApplication::setAttribute(Qt::ApplicationAttribute::AA_UseDesktopOpenGL);
    QApplication app(argc, argv);
    OpenGLWindow w;
    w.show();
    return app.exec();
}

8Observer8 commented 2 days ago

maybeCStringToJsString returns something wrong:

8Observer8 commented 2 days ago

I have tried EMSCRIPTEN_EVENT_TARGET_SCREEN again but it doesn't work now: emcc -g main.cpp -o public/index.html It has the same error as Qt's build. I don't know why it worked before. Maybe I have made something wrong.

main.cpp

#include <emscripten.h>
#include <emscripten/html5.h>
#include <iostream>

int main()
{
    // Mouse Button Down:
    emscripten_set_mousedown_callback(
        "#canvas", nullptr, 0, +[](int eventType,
            const EmscriptenMouseEvent *e, void *userData) -> EM_BOOL
        {
            EMSCRIPTEN_RESULT result = emscripten_request_pointerlock(
                EMSCRIPTEN_EVENT_TARGET_SCREEN, true);
            std::cout << result << std::endl;
            return EM_FALSE;
        });

    return 0;
}

8Observer8 commented 2 days ago

Another trying. Qt has a div element with id="screen". The emscripten_request_pointerlock returns EMSCRIPTEN_RESULT_DEFERRED (1):

main.cpp

#include <QtGui/QMouseEvent>
#include <QtGui/QOpenGLFunctions>
#include <QtOpenGL/QOpenGLWindow>
#include <QtWidgets/QApplication>

#ifdef Q_OS_WASM
#include <emscripten.h>
#include <emscripten/html5.h>
#endif

class OpenGLWindow : public QOpenGLWindow, private QOpenGLFunctions
{

public:

    OpenGLWindow()
    {
        setTitle("OpenGL ES 2.0, Qt6, C++");
        resize(350, 350);
    }

private:

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock("#screen", true);
        qDebug() << result;
    }

    virtual void initializeGL() override
    {
        initializeOpenGLFunctions();
        glClearColor(0.2f, 0.2f, 0.2f, 1.f);
    }

    virtual void paintGL() override
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
};

int main(int argc, char *argv[])
{
    QApplication::setAttribute(Qt::ApplicationAttribute::AA_UseDesktopOpenGL);
    QApplication app(argc, argv);
    OpenGLWindow w;
    w.show();
    return app.exec();
}

8Observer8 commented 2 days ago

I have tried to replace the second argument of emscripten_request_pointerlock from true to false but emscripten_request_pointerlock returns EMSCRIPTEN_RESULT_FAILED_NOT_DEFERRED (-2):

main.cpp

#include <QtGui/QMouseEvent>
#include <QtGui/QOpenGLFunctions>
#include <QtOpenGL/QOpenGLWindow>
#include <QtWidgets/QApplication>

#ifdef Q_OS_WASM
#include <emscripten.h>
#include <emscripten/html5.h>
#endif

class OpenGLWindow : public QOpenGLWindow, private QOpenGLFunctions
{

public:

    OpenGLWindow()
    {
        setTitle("OpenGL ES 2.0, Qt6, C++");
        resize(350, 350);
    }

private:

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock("#screen", false);
        qDebug() << result;
    }

    virtual void initializeGL() override
    {
        initializeOpenGLFunctions();
        glClearColor(0.2f, 0.2f, 0.2f, 1.f);
    }

    virtual void paintGL() override
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
};

int main(int argc, char *argv[])
{
    QApplication::setAttribute(Qt::ApplicationAttribute::AA_UseDesktopOpenGL);
    QApplication app(argc, argv);
    OpenGLWindow w;
    w.show();
    return app.exec();
}

8Observer8 commented 2 days ago

Here is the true case:

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock("#screen", true);
        qDebug() << result;

The problem is here:

var canPerformRequests = JSEvents.canPerformEventHandlerRequests();

canPerformRequests returns 0. How do you think why JSEvents.inEventHandler is 0:

and JSEvents.currentEventHandler is undefined:

8Observer8 commented 2 days ago

But when I pass false here:

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock("#screen", false);
        qDebug() << result;

the JSEvents.currentEventHandler is not undefined:

but SEvents.inEventHandler is 0:

and JSEvents.currentEventHandler.allowsDeferredCalls is undefined:

sbc100 commented 2 days ago

I have tried to replace the second argument of emscripten_request_pointerlock from true to false

Why? Leaving it as true and having EMSCRIPTEN_RESULT_DEFERRED seems like it is working as intended? What is wrong in that case?

8Observer8 commented 2 days ago

Why? Leaving it as true and having EMSCRIPTEN_RESULT_DEFERRED seems like it is working as intended? What is wrong in that case?

I have tried both case: true and false in two posts above. You can see the true case in this post You can see the result there.

sbc100 commented 2 days ago

You said in https://github.com/emscripten-core/emscripten/issues/22133#issuecomment-2196602440 that is returns EMSCRIPTEN_RESULT_DEFERRED.. which would be the correct/working/expected result I think.

sbc100 commented 2 days ago

In deferred mode you don't want to be calling requestPointerLock during the call itself. Instead the call to requestPointerLock is "deffered" until later when will succeed.

Does the differed mode not work for for some reason?

8Observer8 commented 2 days ago

Locking doesn't work. I click in the client area and see 1 in the console every time when I click.

sbc100 commented 2 days ago

I think with the differed mode the actual locking won't happen until the next event. If you click again does the lock take effect?

sbc100 commented 2 days ago

BTW, 1 means EMSCRIPTEN_RESULT_DEFERRED mean the locking will occur on the next event.. so that is form of success.

8Observer8 commented 2 days ago

If you click again does the lock take effect?

No:

request-pointer-lock-clicks

I have uploaded the result build to the free hosting: https://667f439855614296d369a205--small-opengles2-examples-qt6-cpp.netlify.app/

main.cpp

#include <QtGui/QMouseEvent>
#include <QtGui/QOpenGLFunctions>
#include <QtOpenGL/QOpenGLWindow>
#include <QtWidgets/QApplication>

#ifdef Q_OS_WASM
#include <emscripten.h>
#include <emscripten/html5.h>
#endif

class OpenGLWindow : public QOpenGLWindow, private QOpenGLFunctions
{

public:

    OpenGLWindow()
    {
        setTitle("OpenGL ES 2.0, Qt6, C++");
        resize(350, 350);
    }

private:

    void mousePressEvent(QMouseEvent *event) override
    {
        Q_UNUSED(event);

        EMSCRIPTEN_RESULT result = emscripten_request_pointerlock("#screen", true);
        qDebug() << result;
    }

    virtual void initializeGL() override
    {
        initializeOpenGLFunctions();
        glClearColor(0.2f, 0.2f, 0.2f, 1.f);
    }

    virtual void paintGL() override
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
};

int main(int argc, char *argv[])
{
    QApplication::setAttribute(Qt::ApplicationAttribute::AA_UseDesktopOpenGL);
    QApplication app(argc, argv);
    OpenGLWindow w;
    w.show();
    return app.exec();
}

pro

QT += core gui openglwidgets

win32: LIBS += -lopengl32

CONFIG += c++17

wasm: INCLUDEPATH += "C:\emsdk\upstream\emscripten\cache\sysroot\include"

SOURCES += \
    main.cpp

sbc100 commented 2 days ago

So the problem here seems to be that the differed action is not working?

I guess we need to looking to the call to JSEvents.deferCall to see why the deferred call the requestPointerLock is not being made at some later time.

Can you set a breakpoint in the runDeferredCalls function or add some logging there to see why its not being run?

8Observer8 commented 2 days ago

My steps:

I have opened the link: https://667f439855614296d369a205--small-opengles2-examples-qt6-cpp.netlify.app/
I have opened the "Source" tab
I have found runDeferredCalls
I have set a breakpoint inside of runDeferredCalls
I have run an application and make clicks in the client area of the browser but nothing happens (the application is not stopped):

emscripten-core / emscripten

`emscripten_set_mousedown_callback` does not call the callback function from Qt Creator #22133