System wide inertial / kinetic scrolling and rubber banding

[sandreas]

Hello,

thank you for developing the cosmic desktop, I'm pretty impressed so far.

Are you planning to support system wide inertial / kinetic scrolling and rubber banding for touchpad events?

Note: This is a copy of https://github.com/pop-os/cosmic/issues/408, since I was told that this is the wrong repository (for a gnome extension). Feel free to comment on this fact.

Here is an illustration of what I mean, in the front, you see the default behaviour for Apps on linux, in the background there is Firefox with kinetic scrolling enabled, which feels way more natural.

I would love to see this implemented in the Compositor / OS Layer. That's how it is implemented in macOS and it prevents putting too much effort for this on the App developers. It also prevents having different behaviour in different apps because of an app specific implementation...

BTW there is an open source JavaScript implementation by Apple called PastryKit using a magic number momentum * 0.95.

> I am trying to find out the easings/formulas they use and logical conditions they use them in. Apple has a crazy way of writing confusing JS, so even tho i am a full stack developer, i am having a hard time tracing everything down. And i figured few brains is better than one, so lets see, does anyone understand this file? :D > https://github.com/jimeh/PastryKit/blob/master/mobile/dist/PastryKit.js > IN SHORT (so there are no misunderstandings): I am trying to extract a set of physics rules from this file, which i can use as guidelines in order to write my own implementation of scroll on any platform i choose. :) > for example: 'normal' scroll is defined by {>300ms && >10px}, apple uses the following bezier curve when easing the animation of slowdown. cubic-bezier.com/#.25,.46,.1,.94 > UPDATE: We solved this a while ago. We discovered how Apple does it's momentum. https://medium.com/homullus/recreating-native-ios-scroll-and-momentum-2906d0d711ad > After many hours of dissecting the algorithm, we concluded that Apple is in fact using magic numbers. And the magic number is: (drumroll) `momentum * 0.95`. > Basically, while the touch lasts, apple lets you move the screen 1:1. > On touch end Apple would get momentum by dividing number of pixels that the user had swiped, and time that the user has swiped for. If the number of pixels was less than 10 or time was less than 0.5, momentum would be clamped to zero. > Anyways, once the momentum (speed) was known to us, they would multiply it by 0.95 in every frame, and then move the screen by that much. > So idiotically simple and elegant, that it hurts. :)

Since this might be a bit tricky to solve, I would at least implement the following options:

• enable / disable kinetic scrolling completely
• a speed option to control how fast and far the scrolling goes

There is a very detailed discussion over on AvaloniaUI, where I go more into detail and a very helpful response from a very experienced developer.

[LinuxSBC]

I don't think it's possible to do this well at anything lower than the toolkit level. Synaptics tried, but it had a lot of problems like zooming in if you press Ctrl while it's finishing a motion or, if you move the mouse to another window, scrolling that window instead. GTK and Qt have solved this by implementing it at the toolkit level, which would need to be done for Iced. I'm not a COSMIC developer, but maybe it's better to add this information to the issue in libcosmic? https://github.com/pop-os/libcosmic/issues/363

[ph919imperial]

I'm not sure what you mean by "Compositor / OS Layer". On MacOS, kinetic scrolling is handled by the GUI toolkit AppKit, namely NSScrollView. In fact, MacOS native Iced apps do support kinetic scrolling, since winit uses AppKit (see here).

If you mean the input driver as with Synaptics, then that's a bad idea. In general though, there's quite a bit of work that needs to be done upstream before COSMIC has complete gesture support, which it should definitely have. It might be useful to look at GTK's progress.

Please note that this is all my cursory understanding, and I'm not a contributor.

[sandreas]

@ph919imperial

I'm not sure what you mean by "Compositor / OS Layer".

My interpretation of "Compositor Layer" is somewhere between driver and toolkit.

I posted a comment on hackernews with some (hopefully minor) mistakes (like App-Layer instead of Toolkit-Layer), but it makes clear what I mean...

Here is a short summary:

• I know that the driver / libinput is a bad idea. Peter Hutterer pointed that out in very much all of his comments on this
• The GUI Toolkit Layer MAY be the right place, but since Linux supports multiple Toolkits (GTK+, QT, TK, ...) it would be possible, that kinetic scrolling behaves different for different toolkits - not the result I was hoping for
• If it would be implemented in the compositor or window manager (right between driver and Toolkit), it could be implemented in ONE single place and would work for all toolkits at once the same way
• To suplement this, a helping hand could be implemented as an extra libinput-gesture Layer hook providing "synthetic" events for this, which would it make much easier to implement it in the layers above, because the algorithm only takes place there, whereas the code for handling the scroll would just be
  • if (ev->is_synthetic && focus_changed_since_last_nonsynthetic) return;

Important Aspects - Kinetic Scrolling:

• should stop on mouse move
• should NOT stop on Window switch (ALT + Tab)
• should only be executed in the window where it started - independently on multiple instances (e.g. Alt + Tab must not transfer the scrolling events to the new window)
• should stop on Two Finger Tap (if configured)
• should have configurable speed (to match the user's liking)
• should work on background windows

[ph919imperial]

Just to be clear, do you agree that macOS, like the modern wayland stack, implements kinetic scrolling at the toolkit level? The reason why the macOS approach feels universal is because (almost) everyone uses the same apple provided toolkit. Those who have to suffer with Java Swing apps would attest to this.

Regardless, I think your proposal is vague and beyond the purview of COSMIC. If you want input to be handled differently, then you should be raising this with other libinput / w-p members too, since their approach is clear.

Look, as a long-time MacBook user I've tested various Wayland setups over the past few years with an XPS and I fully understand the frustrations you might have. A lot of things are still head-scratchingly broken, like changing scroll speed. Combine that with the absurd defaults I get out of the box and it's near unusable on GNOME. Thankfully system76 devs have that sorted in their designs. Maybe they can go further by allowing custom acceleration profiles too.

[sandreas]

@ph919imperial First I would like to thank you for (deeply!) taking the time to explain. I think you cleared up a very important part by stating that it is the macOS Toolkit, not the OS / Window Manager Layer.

Just to be clear, do you agree that macOS, like the modern wayland stack, implements kinetic scrolling at the toolkit level? The reason why the macOS approach feels universal is because (almost) everyone uses the same apple provided toolkit. Those who have to suffer with Java Swing apps would attest to this.

I can't really agree or disagree, because I just don't know exactly. You're probably right, I never thought about this and I think it's the most interesting statement I read about this for a long time. You really seem to know what you are talking about.

Regardless, I think your proposal is vague and beyond the purview of COSMIC.

It's vague because I probably did not understand everything I should have. However, I thought I'd propose this to someone who does (COSMIC devs) to either explain why or why not implement it in the desktop environment.

Look, as a long-time MacBook user I've tested various Wayland setups over the past few years with an XPS and I fully understand the frustrations you might have.

I'm not frustrated nor unhappy. My Fedora setup with libinput-config just works as I expect, but I just did not understand, why this is such a thing, if there is an open source implementation (Pastrykit), a well known algorithm and even Chromebooks with Linux getting this right. I thought just nobody who could do it would care about it. I can't do it (I would invest a lot of time for the community to get this working decent everywhere) - but I first need to learn about too much stuff (programming language, getting experienced enough, working with Wayland, etc).

A lot of things are still head-scratchingly broken, like changing scroll speed. Combine that with the absurd defaults I get out of the box and it's near unusable on GNOME. Thankfully system76 devs have that sorted in their designs. Maybe they can go further by allowing custom acceleration profiles too.

It's not that bad. If you tinker around you get it working but I agree that sane defaults would solve a lot of things. Thank you, if you have any advice where I should go with this (except the toolkits), I'm ok with it. If it's nothing COSMIC can solve, this issue can be closed.

However if COSMIC could solve it with a small amount of effort, why not? :-)

[ph919imperial]

No problem, I hope my comments and linked sources have been helpful. Again please do note that I am not an expert on this topic. Also, I didn't mean to suggest you were frustrated or unhappy, I was simply detailing my personal experiences.

In terms of how to approach this, we have to be careful to avoid asking XY problems. Especially with Wayland, things can feel excruciatingly slow, and it can be tempting to jump the gun. I think your issue has been really detailed and I thank you for that, and perhaps the focus should first be on what the problems are before looking at possible solutions. From your posts:

It also prevents having different behaviour in different apps because of an app specific implementation...

The GUI Toolkit Layer MAY be the right place, but since Linux supports multiple Toolkits (GTK+, QT, TK, ...) it would be possible, that kinetic scrolling behaves different for different toolkits - not the result I was hoping for

My surmise is that the main issue you're trying to address is progress towards a unified approach to touchpad/screen UX within the linux app ecosystem. To be clear, I agree with you that this is a real and important issue, and I think we're both familiar with the efforts the Linux Touchpad like Macbook project have been making to address this.

To keep this issue on track, there seems to be three main points of concern:

• gesture support
• kinetic/elastic scrolling support
• unified UX between COSMIC and GTK, QT, etc apps

I'll leave it to a contributor to elaborate on any plans to address these. Let me know if I've missed anything.

P.S. Keep in mind that it takes a long time for the entire ecosystem to catch up. Firefox still uses GTK3 and is therefore missing halt gesture support, see the last link in my first post for more details.

[rwjack]

Okay so from a laymans point of view, where do we need to group up and work this out?

[mmstick]

@rwjack Sending a PR to iced-rs/iced to implement the feature in iced.

[rwjack]

https://discourse.iced.rs/t/system-wide-inertial-kinetic-scrolling-and-rubber-banding/442