Fall creators update and New Chrome pen input behaviour Conflict.

[tunepunk]

Problem description

Microsoft introdued "pen scrolling" in Fall creators update, which is the default action for pen (tap and drag) in Edge. Chrome jumped on the bandwagon and added a similar functionality in 62+ builds. In C3 (Desktop Version) this is noticeable when using Pen as input device.

Pen Input Old behaviours (Chrome version 61 and below ) Event sheet: Pen acted as mouse, and could drag and drop events. Sprite editor: Pen acted as mouse and could draw in sprite editor. Layout view: Pen acted as mouse and could manipulate objects.

Pen Input NEW behaviours (in Edge in after fall creators update and Chrome 62+ ) Event sheet: Pen will scroll event sheet like touch, where it should drag and drop events. Sprite editor: Pen will scroll and draw at the same time. (conflict) Layout view: Pen will scroll the layout while trying to manipulate objects.

A lot of softwares have had these issues since fall creators update. Here are some i found listed on github. https://github.com/Microsoft/vscode/issues/36492 https://github.com/aseprite/aseprite/issues/1595

Microsofts official stance is that this is "by design", so this is up to App developers to add specific pen functionality.

Bug filed with Edge: (They don't seem interested to remove pen scrolling) https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/14277341/

Bug Filed with Chrome: (There is option to turn off pen scrolling but disables other pen features) https://bugs.chromium.org/p/chromium/issues/detail?id=788493

Attach a .c3p

N/A

Steps to reproduce

1. Open any project on a Pen enabled device Edge, or Latest Chrome.
2. Test the above mentioned behaviours.
3. Notice conflict or lost functionality.

Observed result

Conflicts or lost behaviour due to new pen input handling in Modern Browsers.

Expected result

Needs code to accomodate pen input and disable pen scrolling where it conflicts.

Affected browsers

• Chrome: yes
• Edge: yes

[tunepunk]

There might be more places where the new pen scrolling will break intended drag and drop features, but I havn't had time to test those. But some places that popped into my mind were. Animation editor, and Project window, (Drag and drop objects to folders and families etc)

I know there may not be many people using pen as main input, so I'm not expecting any high priority on this, but if it's an easy fix to disable pen scrolling where it conflicts (or globally throughout C3) I would be happy :)

[tunepunk]

Using Firefox solves this at the moment, but getting tired to constantly switch browsers every other week because they add some new anti-feature or break pen behaviour. C3, works great with pen in firefox at the moment... lets hope it lasts, but it would be good if you kept pen/touch in mind when designing for desktop. Keep up the good work! <3

[AshleyScirra]

Yeah, it sounds like this is churning quite a lot at the moment, both on the browser and OS side. Hopefully it settles down soon :-\

[tunepunk]

Wanted to add a little update on this one. Edge developers has responded in this bug report, as the new pen changes breaks a lot of things in several web-apps. Not only in construct 3. https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/14277341/

In fall creators update the pen scrolling behaviour is system wide, and will interfere with all web-apps that don't design specifically for pen usage, and differentiate between, pen, mouse and touch.

Hopefully they will add option in future builds to use old pen behaviour but still.... even in 1703, chrome has added their own pen scrolling. So even if microsoft fixes it on their end. Chrome adopted the new behaviour and will break pen usage while using chrome.

I'm guessing I'm just one of very few pen/touch developers here, so not expecting any special case workarounds from your end.

Currently the best option for pen users is to run C3, in Firefox, with build 1703 (pre fall creators update) until they fix the system wide pen scrolling behaviour. Or chrome adds a flag to disable pen scrolling permanently.

I will try to file bug with chrome to highlight conflict with pen scrolling in web apps. Pen scrolling currently breaks any usage of drag&drop, selection rectangles in layout view, object manipulation and sometimes drawing in sprite editor.

[AshleyScirra]

Going to tag this one "difficult" due to the various OS/browser compatibility issues, hardware requirement, and the fact input APIs on the web are a bit of a mess to begin with (especially that Safari doesn't support Pointer Events so we have to use different APIs).

[AshleyScirra]

I think we should wait until the relevant linked browser bugs are resolved before looking in to this further, especially since it appears to affect other apps and not just Construct. Closing for now, please re-file if things aren't working as expected once those browser bugs are closed. Note if other apps exhibit similar problems it's probably not an issue in our code so may not be something we can do anything about. Issues that only appear in Construct and not other apps is what would suggest an issue with our code.

[tunepunk]

These are not actually bugs with the browsers. Chrome has adopted pen scrolling as well (which interfere with drag&drop, object manipulation, etc.) So unless you specifically design your app (ie. construct 3) for use with pen, it will revert to the default behaviour... which now is that pen will scroll and function like a finger, whereas the old pen behaviour it would work as a mouse. That means that I can't use Construct 3 with a pen, not even the desktop version, or other chromium based browser as they all use pen scrolling now.

It seems like they push to the respective app developer to design for pen usage. Otherwize it will revert to default behaviour. (Which is that Pen works as a finger and scrolls)

Microsoft has released a registry hack, for affected users, which allows for Legacy win32 apps to use the old behaviour. Although this does not work with chrome as they have their own pen scrolling built in, so the registry fix has no effect in chrome.

https://www.reddit.com/r/Windowsink/comments/8508fi/controlling_pen_behavior_in_windows_10/

My only option so far, is to use Pre fall creators update build 1703 with firefox, to be able to use pen input att all in Construct 3.

Edge (post fall creators update 1709) - uses pen scrolling as part of Fall creators update, and can't be turned off by the registry fix, as it's not a legacy win32 app.

Edge (prior to fall creators update 1703) - can't run construct 3 as it's requires CSS grid.

Chrome - has their own built in pen scrolling behaviour, so registry fix does not work, disabling pen scrolling in flags, breaks other things, and is deprecated so the flag will even be gone soon.

Firefox - is so far the only option, but is not really a pleasant experience with pen. (Tapping input fields has a quite long delay before it launches the on screen keyboard, and does not shift the page, so the on screen keyboard often places itself on top of the input field so you can't see what you're typing)

Edge is and has always been the best option for pen users before fall creators update. It works perfectly with pen, but C3 requires CSS Grid layout, so I can't run C3 in 1703 version of edge.

So unless you design the web app to support pen. (custom pen behaviour) pen will act as a finger.

Edge or Chrome would be the optimal browsers for pen users, if there was a way to design around the conflict of pen scrolling.

[AshleyScirra]

If pen is handled as touch input then can you not still focus fields and click buttons via taps?

[tunepunk]

The pen don't really revert to a proper touch, and therefore cause a conflict in some cases, and that's why I reported this at with chrome, and edge. Their "new" pen behaviour breaks things that used to work, but they call it a "feature".

Examples of things that break with pen scrolling:

Layout editor using a pen: Drawing a selection rectangle with a pen both scrolls and selects in some cases. Moving & Manipulating objects both manipulates and scrolls in some cases.

Event sheet using a pen: Can no longer drag and drop to reorder events, like you can with a mouse. Editing expressions, becomes a chore, as it works touch, not like a mouse just drag to select text span.

They implemented this new pen behaviour with no regard to functionality in other apps. Early on you couldn't even draw in photoshop as you would scroll instead. I don't know whether microsoft uses an exclusion list for popular apps like photoshop. But browsers such as chrome who implemented their own pen scrolling with no regard for how many web-apps work is no excuse. I've been forced to file bugs with every single app developer where the new pen behaviour is breaking things, as microsoft and chrome does not give a shit, and scrolling with the pen is the new thing.

As you can see in this edge bug report that i filed, there's a huge list of frustrated users that want the old pen behaviour back.

In the feedback hub. A post requesting pen scrolling has 99 upvotes, it got implemented and broke a lot of things. Now there's a post requesting a switch to disable pen scrolling, with some posts dating back over a year, with over 350 upvotes, and microsoft still does nothing, besides providing a registry hack, that may or may not work.

Many Drawing Programs also suffered from this brutal behaviour change, many were forced to design around it. People couldn't even draw any more. They rolled out fixes pretty fast because a lot of people use pen in those apps. Browsers and other apps, where there's just sa small number of pen users, they don't seem to care much.

So, all in all. If app and web-app devs, don't add any special code to accommodate pen users, pen is not a good input choice, where as before it didn't matter. If pen was not really supported, the pen would just behave as a mouse. (drag select text, drag and drop, etc etc)

I'm not sure what the solution is. As construct is designed with touch in mind (works pretty well on mobile even, except you can't drag and drop to reorder events) but pen is like the odd one. It's not a mouse, and it's not touch. It's kind of like a precision finger that should behave like a mouse. At least it should be able to do the same thing as a mouse does.

Sorry for the long posts. I'd gladly donate a pen enabled device for testing purposes, and promote C3 in the pen/tablet user community if there was a quick fix for this.

[AshleyScirra]

If pen used to work like a mouse, then how would you scroll the event sheet view? You cannot do that by mouse input other than the mouse wheel or specifically grabbing the scroll bar. I imagine there's no wheel equivalent on a pen and grabbing the scroll bar would be difficult.

I think one major complication is we are still using touch events or mouse events in the C3 editor, neither of which is designed for pen input, whereas I'd guess browser makers want us to use the pointer events API since it is designed to handle pen input. We already attempted to switch from touch events to pointer events in the past, but it turns out there are some pretty seriously difficult ways that pointer events are not backwards-compatible with touch events - even for basic things like scrolling. So we gave up on that since at the time we thought there was no benefit to following through with the work. If we have to do that to properly handle pen input, then I think given we always have about twice as many things to do than we have time for, then it would be difficult to prioritise a large and complicated job like that just for pen input. So ideally browsers would provide a setting that worked like the old mode, but I can see them resisting that idea...

Anyways we'll probably order in a pen device at some point and give it a spin to see how things work, but like I mention this could be a really difficult thing to change. Additionally I do think there is some merit to just having a wait-and-see approach since especially with a lot of user pressure, browser makers might make some further changes - and it would be a real pain to have to make more changes in Construct again because of that.

[tunepunk]

To answer your first question. If you're using a pen, you still have 5 fingers on the other hand to do scroll, zoom, rotate and other functions, or using the same hand as the pen with extended fingers to do the touch functions. Pen and touch always goes very well hand in hand and complement each other well. But with the new behavior the pen practically becomes an 11th useless finger.

So if I'm getting this right, since pointer events is not being used, there is no way to detect if the input device was a pen. I can understand why that complicates things. I wouldn't wanna ask you to implement a whole new API just for me, and maybe a few other pen users, but maybe something to keep in mind for the future if there's time and resources, because it seems like with these new changes they are trying to push developers to use pointer events instead.

It's a bit sad though, because C2 and C3, is a real pleasure to use otherwise on a tablet like device using pen and touch only. No mouse and keyboard needed, and you can sit on your sofa or in your bed and develop games with a breeze, since there's not much coding involved, and most expressions can be inserted with the expression library.

C3 is still usable with a pen though, but a bit of a chore with the new pen behavior.

By the way. If you don't wanna invest in a windows 10 tablet device, a regular Wacom drawing tablet you can connect to a stationary computer should do the trick as well. Maybe your designer has one already.

Thant's for your reply. Let's hope MS and Chrome come to their senses and remove that abomination (pen scrolling)