Not working on touch screen

[IlyaSemenov]

I am running a full screen Chromely app on a Windows 7 host with a touch screen. For some reason, taps on the screen are not sending any events into the web app running in Chromely (there are no click, no mousedown and no touchstart events emitted). What is even worse, very rarely, like after I tap 100 times, it suddenly passes a few taps as clicks, and then stops working again (I didn't find any pattern, and it's hard to reproduce).

On the same host, when I start desktop Chrome or other desktop apps, touch works normally, I can click buttons, etc.

Clicks with mouse work normally on that Chromely app.

Any advice on how to debug this? I have not a slightest idea on which level could clicks processed incorrectly. .net core? cefglue? chromely itself?

---

UPDATE: to make it clear: for starters, I don't really need the actual 'touch' events (touchstart/etc.), I only need normal clicks to works. Every other desktop app works with the touch screen (taps are registered as clicks), and I highly doubt they all have special built-in support for touch events.

It seems to be the other way around, if the app doesn't even try to handle touches, it works normally using some kind of system wide fallback. And Chromely apparently somehow 'enables' touch support - which apparently disables native tap-to-click conversion - but doesn't actually handle them.

[mattkol]

@IlyaSemenov

Touch screen/multitouch is not a feature that Chromely supports out of the box. I believe it is configurable using CEF command line options.

These links may help: https://stackoverflow.com/questions/28008110/cefsharp-touch-event-handeler https://magpcss.org/ceforum/viewtopic.php?f=7&t=11005

So this may help:

var config = DefaultConfiguration.CreateForRuntimePlatform();
config.CommandLineArgs["touch-events"] = "enabled";

I do not have a touch screen device, so cannot really prove this. If this does not work then, NativeHost may need to be customized - https://docs.microsoft.com/en-us/windows/win32/wintouch/windows-touch-gestures-overview

Keep us posted.

[IlyaSemenov]

@mattkol Thank you for the feedback.

touch-events=enabled didn't make any difference.

What helped, interestingly, was to disable kiosk mode; with config.WindowOptions.KioskMode = false taps are handled as clicks. However, it's not really a solution because I am actually developing a kiosk software.

[IlyaSemenov]

Could it have anything to do with this?

https://github.com/chromelyapps/Chromely/blob/e8d7b172599294404daadc7da1a0ca50a3b7c128/src/Chromely.CefGlue/BrowserWindow/HostBase.cs#L201-L205

https://github.com/chromelyapps/Chromely/blob/4f9943af22807a7ae5f33061bfd01663a760acd4/src/Chromely/Native/WinAPI/WinAPIHost.cs#L511-L514

[mattkol]

@IlyaSemenov it is possible. It was added as part of a PR fix for frameless at the time - https://github.com/chromelyapps/Chromely/pull/84

It may not be needed anymore. It is being cleaned up in current version 5.1, but may still be needed as it is not possible for us to test all scenarios at our end. So with some feed backs we can determine whether we still need it or not.

[IlyaSemenov]

@mattkol I don't think you've actually released version 5.1, have you?

[amaitland]

DoMessageLoopWork would normally be run on a timer and be called on the UI thread 30/60 times per second. With an additional say 10 calls before CefShutdown is called to finish processing any messages. Too many or too few calls to CefDoMessageLoopWork and performance will suffer.

https://github.com/cztomczak/cefpython/issues/245 has some reference links.

[mattkol]

@amaitland thanks for your input. I actually did make a reference to that approach here - https://github.com/chromelyapps/Chromely/issues/230#issuecomment-668834838. The issues are likely related then.

Looks like the On Idle approach was abandoned by CefSharp?

[amaitland]

Looks like the On Idle approach was abandoned by CefSharp

@mattkol Abandoned long ago, example was updated like 4 years ago 😄 calling CefDoMessageLoopWork to infrequently is a problem. No guarantee when idle will be called.

[IlyaSemenov]

I refactored my app to use Chromely 5.1.83-pre01 and here are my observations so far:

• with config.WindowOptions.KioskMode = true; the app hangs the system (the mouse is moving, but it's not reacting on any clicks or keyboard presses, including Alt-F4 and Ctrl-Shift-Esc, so I have to hard reboot). It either doesn't open a GUI window with Chromium, or, the new window is perhaps full screen and completely transparent.
• with config.WindowOptions.Fullscreen = true; the app works good, it opens full screen and taps work as clicks. (I'm yet to test if scrolling works). (UPDATE: drag scrolling also works good!)

This is on Windows 7 32 bit.

[mattkol]

@amaitland got it! Thanks.

[mattkol]

@IlyaSemenov

Which of the base application classes are you using? ChromelyBasicApp ChromelyFramelessApp

[IlyaSemenov]

ChromelyBasicApp, let me try on the other.

[IlyaSemenov]

KioskMode with ChromelyFramelessApp creates a full screen responsive, draggable window, which doesn't render anything. When I move it, it moves whatever was rendered below it. On the photo below, I moved it like 15% to the right and 25% to the bottom.

Anyhow, the original problem is resolved, thank you. I believe I can live with Fullscreen = true just fine.

[mattkol]

@IlyaSemenov

Thanks for looking into it.

So when you say Fullscreen= true works just fine, you mean the touch screen now works? With v5.1 or v5.0?

Also reason you are not seeing anything is this: https://github.com/chromelyapps/Chromely/blob/5e3b2124bccd5f7343ed2c5f72145724c27295ce/src/Chromely/WindowController.Run.cs#L71

That should have been removed too.

[IlyaSemenov]

So when you say Fullscreen= true works just fine, you mean the touch screen now works? With v5.1 or v5.0?

Yes, touch screen works in 5.1 (both taps as clicks, and drag to scroll HTML containers).

[amaitland]

It might be worth a a look at https://magpcss.org/ceforum/apidocs3/projects/(default)/(_globals).html#CefRunMessageLoop() and have CEF run the message loop.

On Windows and Linux you can use the CEF Views implementation to create frameless windows etc. No mac osx support though. https://bitbucket.org/chromiumembedded/cef/issues/1749

[mattkol]

@amaitland running Cef-only message loop on windows is something I have toyed with in the past, but I could not find enough info to see if it is preferred and what windows messaging functionalities may be impacted. But definitely something we could provide as an option.

Thanks.

[amaitland]

@mattkol CefRunMessageLoop is available on win/Linux/mac and is the default for both cefclient and cefsimple sample applications. Cefsimple has a very minimal example see https://github.com/chromiumembedded/cef/tree/master/tests/cefsimple (CEF GitHub mirror, much easier to navigate than bitbucket).

From memory the shutdown steps have to be done in the correct order or you'll have problems, nothing major though.

[mattkol]

@amaitland I just tried swapping Windows messaging with CefRunMessageLoop - it crashed the app. I'll have to find time to look at it a lot deeper. I looked at this at the beginning but I could not remember much of my findings now. I am sure it will work, whether it will impact other features, is what I cannot say. We will see.

Thanks.

[amaitland]

What exactly did you try?

• Call CefRunMessageLoop (making sure Multithreadedmessageloop is false)
• Create form/window in https://magpcss.org/ceforum/apidocs3/projects/(default)/CefBrowserProcessHandler.html#OnContextInitialized()

[mattkol]

@amaitland called CefRunMessageLoop. I think I missed setting Multithreadedmessageloop to false. I need to find time to look at this closer.

Will keep you posted. Thanks.

[mattkol]

@amaitland seems to work great without any impact - https://github.com/chromelyapps/Chromely/commit/df44870ad1928fb95badae7b8c4be7c65f4d5b92. I am making it configurable for now, until we have some feedbacks.

Usage:

Either of these 2 options will turn it on - using CefRunMessageLoop (Windows ONLY)

     var config = DefaultConfiguration.CreateForRuntimePlatform();
     config.WindowOptions.UseOnlyCefMessageLoop = true;

or

     var config = DefaultConfiguration.CreateForRuntimePlatform();
     config.CustomSettings.Add(CefSettingKeys.MULTITHREADEDMESSAGELOOP, "false");

I will add a wiki page soon.

Thanks for your guide.