Inertial scrolling / Kinetic scrolling on linux (Wayland) not working at all

sandreas commented 1 year ago

Describe the bug I'm using Fedora 38 (Gnome + Wayland) and Avalonia 11.0.4. Using an ItemsControl there is no support for inertial scrolling at all - as well as no rubberbanding on top or bottom. Scrolling in general feels very slow and clunky.

Related issues: #9687 and possibly #5195 (for a demo of the expected behaviour)

To Reproduce Steps to reproduce the behavior:

Create an ItemsControl (see below)
Add enough items to make it scrollable
Scroll up and down

<Grid x:Name="Root" RowDefinitions="Auto,*,Auto" Background="Transparent">
        <Grid Grid.Row="0" ColumnDefinitions="*,10,Auto" Margin="80,10,10,10">
            <TextBox Grid.Row="0" Grid.Column="0" Text="{Binding Query}" VerticalAlignment="Center" />
            <Button Grid.Row="0" Grid.Column="2" HorizontalAlignment="Stretch" Command="{Binding SearchCommand}"
                    CommandParameter="{Binding Query}">
                <avalonia:MaterialIcon Kind="Search" Classes="MenuButton" />
            </Button>
        </Grid>
        <ScrollViewer Grid.Row="1">
            <ItemsControl ItemsSource="{Binding SearchResults}">
                <ItemsControl.ItemTemplate>
                    <DataTemplate>
                        <Expander FontSize="28" HorizontalAlignment="Stretch">
                            <Expander.Header>
                                <Grid ColumnDefinitions="3*,5,6*,5,Auto">
                                    <Image Source="{Binding Cover^}" />
                                    <StackPanel Grid.Row="0" Grid.Column="2">
                                        <TextBlock Text="{Binding Title}" />
                                        <TextBlock Text="{Binding Authors}" />
                                    </StackPanel>
                                </Grid>
                            </Expander.Header>
                            <Expander.Content>
                                <TextBlock Text="{Binding Description}" TextWrapping="Wrap" />
                            </Expander.Content>
                        </Expander>
                    </DataTemplate>
                </ItemsControl.ItemTemplate>
            </ItemsControl>
        </ScrollViewer>
    </Grid>

Expected behavior I would expect it to scroll smoothly with inertial support and rubber banding (best case).

Screenshots If necessary, I'll make a short video, but I think the issues linked above show and describe the expected behaviour perfectly fine.

Desktop (please complete the following information):

OS: Linux,
Version: Fedora 38, latest updates installed
Avalonia 11.0.4

maxkatz6 commented 1 year ago

Inertial scrolling is supported only with touch and pen input. Not with a mouse.

sandreas commented 1 year ago

Inertial scrolling is supported only with touch and pen input. Not with a mouse.

I have a touchpad in my Lenovo T480s?! Does that count?

kekekeks commented 1 year ago

Check the output from xinput test-xi2 when scrolling. It's supposed to continue receiving scroll events (Motion event with one of valuators being changed) when you release the touch contact.

maxkatz6 commented 1 year ago

No. Touchpad works as a mouse on any platform and framework. It's not really considered as touch.

But many touchpads produce additional mouse wheel events emulating inertia-like scrolling.

sandreas commented 1 year ago

It's supposed to continue receiving scroll events (Motion event with one of valuators being changed) when you release the touch contact.

But many touchpads produce additional mouse wheel events emulating inertia-like scrolling.

@kekekeks @maxkatz6 I might be wrong but I think this is only true on X11 Systems with e.g. synaptics driver. For wayland with libinput, this is not the case.

No. Touchpad works as a mouse on any platform and framework. It's not really considered as touch.

I understand your point, but in my opinion it IS a touch device. On macOS inertial scrolling is enabled by default and every app also supports rubberbanding (Avalonia 11 also does without taking care of it!), because it is implemented system wide in a clean way.

I made a short video, the window in the back is Firefox supporting inertial scrolling and in the front it is ToneAudioPlayer - my current Avalonia project, feeling very clunky:

Unfortunately inertial scrolling via Touchpad on Linux has a long and sad history - probably I'm one of a few persons interested on this and no one else cares ;-) Here is an interesting overview "Linux touchpad like a Macbook" mainly for gesture support.

Inertial scrolling in X11

Regarding X11 (old style) with e.g. the Synaptics driver, the driver itself provided a workaround, that fires additional scroll events to simulate inertia-like scrolling. One problem with this was, that if you pointed to a window and scrolled, and accidentaly moved your mouse to another window, the scroll continued in the other window, which was also seen as unwanted behaviour and "unfixable" because the events could not be window specifc. Don't get me wrong, this workaround was OK-ish, but far from clean.

In Wayland

In Wayland, things changed a bit. The main developer of libinput (Peter Hutterer) rejected a driver implementation long time ago, because it is the wrong place, he also published a more recent article about libinput with a short scrolling paragraph . There is no clear suggestion or decision, how and where to implement this, except that the current situation is: App developers have to implement this themselves - while some toolkits (GTK, QT) provide helpers to do it "right", see Dolphin (KDE FileManager) .

My thoughts

In my opinion this is still questionable, because this leads to different implementations and UX on the same system by different Apps. I think this should perhaps be implemented in the display server / window manager (for GNOME it is MUTTER) to provide a consistent UX, so it works out of the box in every App without taking care of it (like on macOS).

There is also discussions about too high inertial scrolling speed, which is NOT configurable in any way by default in GNOME. That's why someone implemented libinput-config, which is pretty much a hack, that hooks into the event stream and depending on a config file it removes events or calculates extra events to change the speed (and other settings).

However, I'm not an expert but it turns out that inertial scrolling works out of the box on some distros / Apps (e.g. Fedora with Firefox), but does not with others.

What about Avalonia?

For Avalonia as a cross platform solution I would love to see an expert taking a look on this for Linux because it would improve the Look & Feel of EVERY Avalonia App out there and it would make Avalonia even more attractive for Linux UI developers, because it would be one of the few frameworks supporting this.

Focusing on Wayland would be OK in my opinion, because wayland is pretty likely the future.

I have no idea what to do about having a Framebuffer App (e.g. for Raspberry) and using a keyboard with integrated touchpad like the Logitech K400... maybe this would have to be implemented in Avalonia and enabled via user config... whatever.

Apples algorithm as Open Source in JavaScript

If you would like to implement inertial scrolling the hard way, like:

<ScrollViewer SimulateInertialScrolling="True"></ScrollViewer>

there is an Open Source (?) JavaScript implementation by Apple called PastryKit, which is discussed in this stackoverflow thread.

kekekeks commented 1 year ago

I believe the linked Dolphin implementation handles touch events from touchscreen (QTouchEvent), not from touchpad.

Please, run xinput test-xi2 to check what events are actually being delivered.

kekekeks commented 1 year ago

You can also use https://github.com/jwrdegoede/wev to check raw wayland events

sandreas commented 1 year ago

Please, run xinput test-xi2 to check what events are actually being delivered.

@kekekeks Here's the output of both:

Xinput dump: xinput-dump.txt

Wev dump: wev-dump.txt

kekekeks commented 1 year ago

Your scroll events are delivered as axis events (aka mouse scroll), not as touch events in both dumps.

Since synaptics emulated inertial scrolling is delivered in the same way it's be indistinguishable from normal scroll.

sandreas commented 1 year ago

Your scroll events are delivered as axis events (aka mouse scroll), not as touch events in both dumps. Since synaptics emulated inertial scrolling is delivered in the same way it's be indistinguishable from normal scroll.

Thank you very much, your response times are way better than I can keep up with :-) That's impressive... you're really taking care. Unfortunately, I'm not totally sure how to interpret your answer...

Basically you are telling me, that we cannot or even should not fix this in AvaloniaUI, right?

If so, before closing this issue, I would really appreciate a little hint, where to go next (also for documentation purposes helping other Avalonia users with this problem):

Is the touchpad driver is acting in a wrong way (e.g. libinput sends axis events where touch events should be sent when using two finger scroll)?
Or is the hardware not recognized the right way? (e.g. wrong touchpad capabilities in the Kernel)
Or is it a config problem? (since libinput has no configuration file in Wayland besides the existing system options in GNOME or gnome-tweaks, I could not do much about this, except patching and recompiling the whole thing)

Here is a config dump (some options seem to be reported wrong, see comments):

libinput list-devices

Device:           SynPS/2 Synaptics TouchPad
Kernel:           /dev/input/event4
Group:            6
Seat:             seat0, default
Size:             96x60mm
Capabilities:     pointer gesture
Tap-to-click:     disabled              # enabled via gnome-twaks, no idea, why this is reported incorrectly
Tap-and-drag:     enabled
Tap drag lock:    disabled
Left-handed:      disabled
Nat.scrolling:    disabled              # enabled via settings, no idea, why this is reported incorrectly
Middle emulation: disabled
Calibration:      n/a
Scroll methods:   *two-finger edge      # star shows selected option
Click methods:    *button-areas clickfinger             
Disable-w-typing: enabled
Disable-w-trackpointing: enabled
Accel profiles:   flat *adaptive custom
Rotation:         n/a

kekekeks commented 1 year ago

1) libinput converts two-finger gesture intoLIBINPUT_EVENT_POINTER_AXIS events that only contain scroll delta and that's expected behavior (you can see that that by running libinput debug-events). 2) Wayland compositor (or Xorg server configured to use xserver-xorg-input-libinput) receives such scroll delta from libinput 3) (wayland-only) Wayland compositor passes that scroll event to XWayland 4) Xorg (XWayland) adds the scroll delta to it's accumulated scroll value (valuator) and generates XI_Motion event 5) X11 app (such as Avalonia) compares the valuator value to a previously saved value and gets back the scroll delta

A native wayland client would just receive scroll delta without having to deal with valuators ( wl_pointer-related event with axis/diff).

With synaptics driver it's mostly the same, except it generates fake scroll events for inertial scrolling.

However neither the window server nor the application have a clue of what's going on with finger movements or whether inertial scrolling is auto-provided by whatever component is handling raw evdev events.

The only thing we know about the device is some description reported by XInput:

description

``` ⎜ ↳ DELL0831:00 044E:1220 Touchpad id=14 [slave pointer (2)] Reporting 8 classes: Class originated from: 14. Type: XIButtonClass Buttons supported: 12 Button labels: "Button Left" "Button Middle" "Button Right" "Button Wheel Up" "Button Wheel Down" "Button Horiz Wheel Left" "Button Horiz Wheel Right" None None None None None Button state: Class originated from: 14. Type: XIValuatorClass Detail for Valuator 0: Label: Rel X Range: 0.000000 - 1199.000000 Resolution: 12000 units/m Mode: relative Class originated from: 14. Type: XIValuatorClass Detail for Valuator 1: Label: Rel Y Range: 0.000000 - 635.000000 Resolution: 12000 units/m Mode: relative Class originated from: 14. Type: XIValuatorClass Detail for Valuator 2: Label: Rel Horiz Scroll Range: 0.000000 - -1.000000 Resolution: 0 units/m Mode: relative Class originated from: 14. Type: XIValuatorClass Detail for Valuator 3: Label: Rel Vert Scroll Range: 0.000000 - -1.000000 Resolution: 0 units/m Mode: relative Class originated from: 14. Type: XIValuatorClass Detail for Valuator 4: Label: Abs MT Tool Type Range: -1.000000 - -1.000000 Resolution: 0 units/m Mode: relative Class originated from: 14. Type: XIScrollClass Scroll info for Valuator 2 type: 2 (horizontal) increment: 37.000000 flags: 0x0 Class originated from: 14. Type: XIScrollClass Scroll info for Valuator 3 type: 1 (vertical) increment: 29.000000 flags: 0x0 ```

As you can see, it doesn't tell us if it handles inertial scrolling by itself. Wayland's wl_pointer provides even less info.

So at least with X11 we simply don't have information to decide if we should handle inertial scrolling.

We could implement inertial handling for Wayland (and probably XWayland) if we'd know for sure that Wayland never handles inertial scrolling by itself.

sandreas commented 1 year ago

@kekekeks Wow, thanks for this detailed insight.

We could implement inertial handling for Wayland (and probably XWayland) if we'd know for sure that Wayland never handles inertial scrolling by itself.

Well, maybe it could be a solution to provide this feature as opt-in... something like:

TopLevel.GetTopLevel(mainControl)?.PlatformSettings.InertialScrollingOptions = new InertialScrollingOptions() {
    Enabled = true,
    Factor = 0.2
};

This way you could provide a SettingsView / SettingsViewModel in your App, where the user can enable inertial scrolling, when he is on Linux... If one day Wayland comes up with a working solution, he can just disable it and it still works... :-) On other systems these options could simply be hidden or grayed out. The big advantage would be that Avalonia would provide a helper but does not implement too much detail, so the support for this would mainly take place by the App developers and not by the Avalonia team.

But I don't wanna waste your time with this if it is too much effort... Maybe with some helpful instructions / guidance I could implement this myself and submit a PR.

The settings for libinput-config are these, but I don't thing it is neccessary to implement them all:

override-compositor={disabled,enabled}
tap={disabled,enabled}
tap-button-map={lrm,lmr}
drag={disabled,enabled}
drag-lock={disabled,enabled}
accel-speed=[number]
accel-profile={none,flat,adaptive}
natural-scroll={disabled,enabled}
left-handed={disabled,enabled}
click-method={none,button-areas,clickfinger}
middle-emulation={disabled,enabled}
scroll-method={none,two-fingers,edge,on-button-down}
scroll-button=[number]
scroll-button-lock={disabled,enabled}
dwt={disabled,enabled}
scroll-factor=[number]
scroll-factor-x=[number]
scroll-factor-y=[number]
discrete-scroll-factor=[number]
discrete-scroll-factor-x=[number]
discrete-scroll-factor-y=[number]
speed=[number]
speed-x=[number]
speed-y=[number]
gesture-speed=[number]
gesture-speed-x=[number]
gesture-speed-y=[number]
remap-key=[number]:[number]

kekekeks commented 1 year ago

Well, maybe it could be a solution to provide this feature as opt-in... something like:

The scrolling will be completely broken if system compositor already provides events with scroll inertia. We need a way to detect that somehow.

sandreas commented 1 year ago

The scrolling will be completely broken if system compositor already provides events with scroll inertia. We need a way to detect that somehow.

Aah I see where this is going. Having inertial scrolling enabled in an Avalonia App by custom user setting and then upgrading the compositor which newest features natively support inertial scrolling by default will make the app unusable under specific circumstances.

This is really bad... Unfortunately I can't think of any possibility to detect inertial scrolling events in a reliable way - especially when the detection is performed via event analysis on the fly. Since the inertial events are not distinguishable from non-inertial ones other than "looking at the accelleration curve", I can't even think of a solution I would not consider as dirty hack.

What I can think of (non of these solutions are recommendable in any way):

Detecting scrolling events and categorize them as inertial via algorithm => unreliable / dirty hack (please correct me if I'm wrong)
Looking up the system settings / environment variables / compositor features => impossible atm, because it is not implemented or configured in any current distro, there is no specification for this and not even planned afaik
Whitelisting the versions / systems, that DON'T support it => pointless, because it would be a lot of effort to keep this database up to date
Blacklisting the versions that support it => maybe, but still have the same problems of managing a database of supported / unsupported versions and it won't work for already rolled out apps
Proposing and submitting a Wayland protocol / Compositor design change, where inertial events are marked somehow to make them distinguishable => this should not be Avalonia's goal, there is enough other work to do and also I'm pretty sure it would be hard to get accepted a specs change

However, just for sake of completeness, I found a library written in JavaScript, that claims it can detect inertial scrolling, but I'm not convinced it works like expected.

https://github.com/d4nyll/lethargy

@kekekeks Thank you very much for supporting me on this, but now enough of wasting your time. Please let me know if I should close this bug as wontfix or leave it open in case anyone comes up with a revolutionary idea that makes implementing this possible in a clean and recommendable way...

kekekeks commented 1 year ago

then upgrading the compositor which newest features natively support inertial scrolling by default will make the app unusable under specific circumstances.

I suspect it's already the case for WSL and nested wayland sessions that are running on top of Xorg with synaptics driver.

The linked JS library seems to detect all scroll events as inertial on my machine.

We might provide some toggle to be set by the app if said toggle is strictly user-configurable.

sandreas commented 1 year ago

We might provide some toggle to be set by the app if said toggle is strictly user-configurable.

+1 for that. It could also be an environment variable to be able to enable/disable it without having to start the app (with potentially broken user interface). I think the implementation of this in Avalonia based on PastryKit could be a fun new experience, but I personally would not know where to start. Let me know if I can help you with the implementation.

AvaloniaUI / Avalonia