search

OpenBoard-org / OpenBoard

OpenBoard is a cross-platform interactive whiteboard application intended for use in a classroom setting.
https://openboard.ch/
GNU General Public License v3.0
2.4k stars 429 forks source link

Cannot Interact (Click on Buttons) with Applications [Question] #942

Open IRCCRI opened 7 months ago

IRCCRI commented 7 months ago

Not sure if I'm using the right feature, but sometimes I cannot press buttons on with certain applications (the stopwatch especially). I have tried using the "Select and Modify Objects" feature and the "Interact with Items" feature, but the application does not respond. For some reason, other applications (Triangle, Axes) work with those features. I haven't tested all the applications, but it's the stopwatch that I really want to use and it's that application that I cannot interact with.

letsfindaway commented 7 months ago

I cannot reproduce this here, so please provide more information about you system and environment. e.g. answer the following questions (edit the proposed answers to reflect your situation):

IRCCRI commented 7 months ago

Sorry, I should have mentioned this in my initial post. This error occurs on a touchscreen—it occurs when I touch that touchscreen with my finger while either the "Select and Modify Objects" feature or the "Interact with Items" feature are selected, but not when I use my mouse. Anyway, here is the relevant info. reproducibility : systematic (somewhat. The stopwatch doesn't work for sure, and this is the app I care about the most) OpenBoard version bug happens with : 1.7.0 bug doesn't happen with : n/a didn't check : all other versions of OpenBoard OS : bug happens with : Windows 10 Home 22H2 bug doesn't happen with : n/a didn't check : Linux macOS Installation source : OpenBoard download page

letsfindaway commented 7 months ago

it occurs when I touch that touchscreen with my finger while either the "Select and Modify Objects" feature or the "Interact with Items" feature are selected, but not when I use my mouse.

That's interesting.

bug happens with : Windows

and your platform.

I just tested this on my openSUSE Linux and here the Stopwatch works with mouse, touch and pen. So it seems to be Windows specific. But I have to pass on that one. I don't have Windows and cannot further investigate. But now you gave the relevant info and perhaps can somebody else jump in and analyze this further.

sebojolais commented 7 months ago

@IRCCRI by a touchscreen do you mean an IFP (Interactif Flat Panel) connected as an external screen ?

If yes, i will provide some details here about similar experience. If no, I will open a new ticket.

IRCCRI commented 7 months ago

Added my Windows platform and version.

@IRCCRI by a touchscreen do you mean a IFP (Interactif Flat Panel) connected as an external screen ?

If yes, i will provide some details here about similar experience. If no, I will open new ticket.

It doesn't work with both an IFP connected as an external screen and with my laptop's touchscreen monitor. You should provide your details as your problem seems to be related to mine. Are you having trouble getting the stopwatch to work too?

BTW, I have tried all the apps and here is what does not work. By this I mean I cannot interact with those apps using my finger, the Select and Modify Objects feature or the Interact with Items feature, and my laptop's touchscreen monitor.

sebojolais commented 6 months ago

Hi,

I got some troubles with OB 1.7 and an IFP used as interactive external screen. My setup was Linux on X11.

Everything was fine if the screens are in duplicated mode. If the IFP is setup as an extension of the desktop, the interactivity on the IFP does not work but it works on the laptop with the mouse.

And I am in contact with someone with interactive issue with OB 1.7, Windows and IFP from Promethean and Viewsonic. And lot of classroom are affected and they can not switch to OB 1.7 due to this issue.

sebojolais commented 6 months ago

Tests of a Speechi IFP on Windows

Because I heard some troubles with IFP, Windows and OB 1.7, I did some tests on Windows also and the results are below.

A laptop is connected to an IFP used as an external screen and the touch information stream over an USB port. The IFP is a Speechi model: ST-55-HD-AND-002 The interactive actions with finger work fine without OB. I can select file or shortcut on the desktop. And OB works always as expected from the laptop point of view with the mouse.

Windows 10 - OB 1.7.0

In duplicated screen mode :

In this mode, the interactive feature of the IFP works fine on the menu of OB (Stylus bar, main bar with color, left and right panels). There are issue on applications in Board mode.

These apps are working as expected:

These apps do not work on the IFP:

In extended desktop :

Every apps do not work on the IFP:

Windows 10 - OB 1.6.4

In duplicated screen mode :

Every apps are working as expected:

In extended desktop :

Every apps do not work on the IFP:

Conclusion

With the Speechi IFP (model: ST-55-HD-AND-002) on Windows 10 ,

In duplicated screen, OB 1.7.0 introduces interactive issues with these apps : Calculator, Camera, ColorPicker, GeoInfo, GoogleMaps, Papier

It seems that this IFP on Windows 10 has always interactive issues in extended desktop mode, on OB 1.6.4 and 1.7.0.

I do not understand well why the finger actions on the IFP are not received everytime by OB 1.7. Any idea on this issue is welcome. And any idea on the way to go deeper on the root cause of this issue is really welcome.

letsfindaway commented 6 months ago

To add some information: Interactivity on the extended display is currently not implemented. See https://github.com/OpenBoard-org/OpenBoard/issues/838#issuecomment-1840694463 for the answer to a feature request some time ago.

For the other applications: Those which are reported to work (from Ruler to Cache) are C++ widgets. They directly receive the events from Qt. All others are web widgets. This means they are some kind of embedded web browser window with JavaScript code for implementation of these apps. Here Qt has to forward the events to the embedded web browser.

One of the most fundamental changes from 1.6.4 to 1.7.0 was the exchange of the browser engine used. While this was WebKit before, we now use WebEngine, which is based on Chrome. WebKit is no longer supported by Qt, so this change was necessary. It may be that under some circumstances the forwarding of touch events to the browser does not work.

@sebojolais: You also mentioned that you had the same issue under Linux/X11. Can you give more information about this?

I have not analyzed that further but just as a further point: Originally, mouse events and touch events are quite different and they arrive at the application as distinct events. But as one typically expects that a short touch with a single finger acts the same as a mouse click, there is a built-in mechanism in Qt: If the touch event is just a single finger and for a short time, and it is not handled by the application, then it is converted to a mouse click event and offered again to the application.

May be there are some difficulties with that in specific circumstances.

sebojolais commented 6 months ago

The interactive issue on linux/x11 were only in extended desktop mode. Now, if this feature is not implemented, I can say I did not see any other issue.

So now I can say the interactive issue is on Windows, with OB 1.7.0 and javascript applications. @letsfindaway thank you for your answer :+1:

sebojolais commented 6 months ago

I did more tests:

  1. On Windows 10, on OB 1.7, on an IFP, the interactive events arrive in this method :

    void UBGraphicsWidgetItem::mousePressEvent(QGraphicsSceneMouseEvent *event)
{
}

    But they are not sent to the javascript code of the application.

  2. I tried to set more settings in UBWebController with no more success: 

    UBWebController::UBWebController(UBMainWindow* mainWindow)
{
....
settings->setAttribute(QWebEngineSettings::TouchIconsEnabled, true);
settings->setAttribute(QWebEngineSettings::FocusOnNavigationEnabled, true);
....

  3. I tried to set an additional option in UBGraphicsWidgetItem constructor with no more success: 

    UBGraphicsWidgetItem::UBGraphicsWidgetItem()
{
....
setAcceptTouchEvents(true);
....

Conclusion

No good news in this comment. The idea of a bug inside Qt becomes close. Next step: I would like to test with the last Qt6 to see if we observe the same behaviour. At the end, a minimal test program that reproduce the issue should be helpfull.