No touch events in BlazorGL project

[dtaylorus]

My BlazorGL project receives mouse input but receives no touch events and is therefore unusable on an iPhone.

I'll investigate further today to provide more specifics.

What version of KNI does the bug occur on:

• KNI 3.13.9001

What operating system are you using:

• Windows 11/Chrome, iOS 17.6.1/Safari

What KNI platform are you using:

• BlazorGL

[dtaylorus]

The condition

if ('ontouchstart' in window)

in Window.8.0.1.js returns false on Windows 11/Chrome so the touch events aren't registered.

[dtaylorus]

window.addEventListener('touchstart', ... is working where window.ontouchstart = ... is not.

[dtaylorus]

Next up TouchPanelStrategy.DisplayWidth/Height in TouchPanelStrategy.Legacy.cs are both 0 in the AddPressedEvent method so TouchLocationData._position is always (0, 0).

[dtaylorus]

I submitted a pull request for fixing the touch events on Chrome at https://github.com/nkast/Wasm/pull/6.

There's still a fix required for the TouchPanel DisplayWidth/Height TBD, and there appears to be an issue with the fake touch event id's that is invoking a lot of debug assertions.

[dtaylorus]

Confirmed that setting TouchPanel DisplayWidth/Height to the back buffer size fixes touch input. Just need to figure out the correct place to hook in the update since I currently just hacked it in a bunch of places.

[nkast]

The best place to set it is at the end of OnApplyChanges() overload, after PresentationParameters are set for backbufferWidth/Height. https://github.com/kniEngine/kni/blob/a22df9536ad7a6028d5e5929bd795e2658915755/Platforms/.Blazor/ConcreteGraphicsDeviceManager.cs#L80

Android set it at GameWindow.ClientBoundsChanged(...). It works, but it's not quite right.

iOS had the right idea here... https://github.com/kniEngine/kni/blob/a22df9536ad7a6028d5e5929bd795e2658915755/Platforms/.iOS/ConcreteGraphicsDeviceManager.cs#L158

The other platforms set an event on DeviceReset, but that is not yet implemented. And they never supported touch anyway. There is no reason to dispatch an event internally, we can simply set the values right after the backbuffer dimessions are set, at the end of ApplyChanges().

[dtaylorus]

Thanks. That's one of the places I have it now. I'll remove the other places I have it and test it tomorrow and send you a pull request.

[nkast]

Maybe we also have to update it on window resize, I have to check.

[dtaylorus]

Yeah, OnApplyChanges is not called from CreateDevice so the touch panel display size isn't initialized. The only place I see that's call by both CreateDevice and OnApplyChanges is DoPreparingDeviceSettings, so I put the touch panel update there. It probably makes GraphicsDevice_DeviceReset_UpdateTouchPanel redundant but I didn't remove that.

Also, there's one other small change in TouchPanelStrategy.Legacy.cs where we need to convert one of the int's to a float before dividing (or move the parens) to be sure we're doing floating point division rather than integer division, e.g.,:

position.X = position.X * ((float)DisplayWidth / winSize.X);

I'll submit a pull request shortly.

[nkast]

Here are some more details,

GraphicsDevice during initialization will override the PresentationParameters.BackBufferWidth/Height. Because we don't have the power to resize the browser window, the PresentationParameters passed down from GraphicsDeviceManager are ignored. https://github.com/kniEngine/kni/blob/main/Platforms/Graphics/.BlazorGL/ConcreteGraphicsDevice.cs#L61-L66

The GameWindow calls ApplyChanges() when browser is resized, to update the GraphicsDevice.PresentationParameters. https://github.com/kniEngine/kni/blob/main/Platforms/Platform/Blazor/BlazorGameWindow.cs#L274-L277

So I think the best place to place a call to UpdateTouchPanel(...) is somewhere after the device is created, after a call to CreateDevice(gdi); https://github.com/dtaylorus/MonoGame/blob/f3f759b4de6e5f28c8cc2a4e73110a7470cf2946/MonoGame.Framework/GraphicsDeviceManager.cs#L116 or inside that method at the end.

Also a call is needed at the end of ApplyChanges(), after the call to GraphicsDevice.Reset(gdi.PresentationParameters);

Also the input must always come from GraphicsDevice.PresentationParameters, because that's holding the actual backbuffer /window size.

The fix in TouchPanelStrategy is correct, I broke that when refactoring the TouchPanel.

[nkast]

I see that the Reset event is actually wired. This is an issue with initialization. On Chrome emulator touch work after refresing the page, and resizing. So perhaps we only need to set the TouchPanel in CreateDevice(). The integer division just works because it's always 1.

Meanwhile, I will investigate why the events aren't registering on the window from the start.

[dtaylorus]

The events not registering on the window are fixed by https://github.com/nkast/Wasm/pull/6

[nkast]

I found what is going on. Desktop browsers don't have touch support. It's only when I open Dev-tools and enable the mobile device emulator that touch events are presents, that's why I needed to refresh the page.

That also address part of your issues. You mention that events are not registered on Windows 11/Chrome. That is by design. If you have a touch screen, any events will be emulated as mouse events. That is also what will happen in mobile that doesn't support touch. The events will appear as Mouse events.

If fix the issue with the initialization of TouchPanel, by placing it right after the GraphicsDevice creation #1837. I've tested it on an android device and it's working. The update method on the DeviceReset is firing just as excepted. I remembered why I put it there. Reset() can also be called by the user, not just from inside ApplyChanges().

[nkast]

I am going to review your wasm change next. Do you have any theory why those changes are needed in iOS?

According to the documentation, any event should be available via a property, or AddEventListener, and the property is the only way to check if the feature exists. So, is this a bug in safari, or a known behavior?

[dtaylorus]

Touch events work great in Chrome on Windows 11 with my fix in your Wasm project dependency nkast/wasm#6. No need for developer mode.

If you have a touchscreen laptop browse to https://whamblegame.com/beta to try it for yourself. (There are a bunch of platform specific things I still need to port, but the basic functionality is there including touch in the browser.)

---

From: Nikos Kastellanos @.> Sent: Sunday, September 8, 2024 7:31:31 AM To: kniEngine/kni @.> Cc: David Taylor @.>; Author @.> Subject: Re: [kniEngine/kni] No touch events in BlazorGL project (Issue #1820)

I found what is going on. Desktop browsers don't have touch support. It's only when I open Dev-tools and enable the mobile device emulator that touch events are presents, that's why I needed to refresh the page.

That also address part of your issues. You mention that events are not registered on Windows 11/Chrome. That is by design. If you have a touch screen, any events will be emulated as mouse events. That is also what will happen in mobile that doesn't support touch. The events will appear as Mouse events.

If fix the issue with the initialization of TouchPanel, by placing it right after the GraphicsDevice creation #1837https://github.com/kniEngine/kni/pull/1837. I've tested it on an android device and it's working. The update method on the DeviceReset is firing just as excepted. I remembered why I put it there. Reset() can also be called by the user, not just from inside ApplyChanges().

— Reply to this email directly, view it on GitHubhttps://github.com/kniEngine/kni/issues/1820#issuecomment-2336650247, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAS4U7YS6434FZHEM4FJUDLZVQYRFAVCNFSM6AAAAABNUM2VUOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZWGY2TAMRUG4. You are receiving this because you authored the thread.Message ID: @.***>

[dtaylorus]

I am going to review your wasm change next.

Do you have any theory why those changes are needed in iOS?

According to the documentation, any event should be available via a property, or AddEventListener, and the property is the only way to check if the feature exists. So, is this a bug in safari, or a known behavior?

I read that Chrome simply doesn't expose those touch events in the DOM as properties even though they exist via AddEventListener. It's not just on iOS. They're needed on Chrome for Windows as well. Apparently it's a well-known issue with Chrome.

[dtaylorus]

Actually, you may be right that the issue on Windows/Chrome doesn't require the Wasm fix. It's been a while and I don't remember which fix actually made touch on Windows work and it may have been the one you fixed.

But yes, there seems to be a bug on iOS that needs the fix in the Wasm project. Both for Safari and Chrome, which I guess makes sense because Chrome on iOS is really just a wrapper around Safari.

[nkast]

I just went to the game you linked. Using Windows 10/Laptop with touchscreen.

You are right, it does work! Both firefox and Chrome. On dev console I can see that window.ontouchstart is not defined, but window.navigator.maxTouchPoints returns 10. So confusing.

The mozilla page on touch, says that 'Touch events are typically available on devices with a touch screen, but many browsers make the touch events API unavailable on all desktop devices, even those with touch screens.'.

I misread that as, Touch functionality is not available on Desktop, while they meant that the event properties are not available.

I was testing your changes on an Android, everything fine there, nothing is broken. Also, on chrome, I can switch between the emulator back and forth, without refreshing the page. Which definitely is an improvement.

[dtaylorus]

Awesome. Thanks for getting to the bottom of this. Great to have a fully functional port of a MonoGame project on the web!