ScrollContainer on RPI touchscreen

[Michael-F-Ellis]

tl;dr Need a way to "fake" scroll input to a ScrollContainer

My app is targeted to the "official" Raspberry Pi touchscreen. AFAICT it is not possible to arrange for that touchscreen to accept a ScrollMethod option to libinput. Using a mouse or other external input device is not a possibility since it will be installed in semiconductor fabs and similar facilities that forbid keyboards and mice.

One of the pages in the app has a log file displayed in a ScrollContainer. I've verified that scrolling works if I connect a mouse with a scroll wheel but finger dragging has no visible effect.

Is there a way to force a scrollbar to appear and have it respond to touches?

Failing that, is there a way to send scroll position changes from a pair of button widgets?

Thanks!

[andydotxyz]

Good questions! really what we need to do is identify that there is a touchscreen installed so that the tollkit can handle the events correctly. I'm not sure how we would do that.

Second to that a (very accurate) tap and drag on the scroll bar will work, but is not ideal.

Third option like you say is with buttons. Rather than sending scroll events you can handle the ScrollContainer.Offset value directly - updating it then calling Refresh() should work just fine.

[Michael-F-Ellis]

Thanks, Andy.

I installed libinput-tools and captured some events from touchscreen. My app was running at the time and responding to single touches.

Touches produce a TOUCH_DOWN, TOUCH_UP pair. Swipes produce TOUCH_DOWN, TOUCH_MOTION, ... TOUCH_UP -- exactly as one would expect.

If I knew how to access the touch events in the context of a ScrollContainer, I could try writing a handler that changes ScrollContainer.Offset in proportion to the changes in the y-coord of successive TOUCH_MOTION events.

Does fyne make the events accessible in any way or is there some intervening layer that filters out the raw events making them inaccessible?

*Here's the command invocation and the reported mapping on /dev/input/event3. FT5406 is the touchscreen.

pi@hmi0:~ $ sudo libinput debug-events
-event3   DEVICE_ADDED     FT5406 memory based driver        seat0 default group2  cap:t ntouches 10 calib

Here's a single touch report

 event3   TOUCH_DOWN       +32.50s  0 (0) 31.84/51.35 (255.00/247.00mm)
 event3   TOUCH_FRAME      +32.50s
 event3   TOUCH_UP         +32.55s
 event3   TOUCH_FRAME      +32.55s

and here's upward swipe

 event3   TOUCH_DOWN       +86.44s  0 (0) 34.33/86.49 (275.00/416.00mm)
 event3   TOUCH_FRAME      +86.44s
 event3   TOUCH_MOTION     +86.46s  0 (0) 34.33/86.69 (275.00/417.00mm)
 event3   TOUCH_FRAME      +86.46s
 event3   TOUCH_MOTION     +86.47s  0 (0) 34.33/86.49 (275.00/416.00mm)
 event3   TOUCH_FRAME      +86.47s
 event3   TOUCH_MOTION     +86.52s  0 (0) 34.33/86.07 (275.00/414.00mm)
 event3   TOUCH_FRAME      +86.52s
 event3   TOUCH_MOTION     +86.54s  0 (0) 34.33/84.82 (275.00/408.00mm)
 event3   TOUCH_FRAME      +86.54s
 event3   TOUCH_MOTION     +86.56s  0 (0) 34.21/83.37 (274.00/401.00mm)
 event3   TOUCH_FRAME      +86.56s
 event3   TOUCH_MOTION     +86.57s  0 (0) 34.21/81.50 (274.00/392.00mm)
 event3   TOUCH_FRAME      +86.57s
 event3   TOUCH_MOTION     +86.59s  0 (0) 34.08/79.63 (273.00/383.00mm)
 event3   TOUCH_FRAME      +86.59s
 event3   TOUCH_MOTION     +86.61s  0 (0) 33.96/77.75 (272.00/374.00mm)
 event3   TOUCH_FRAME      +86.61s
 event3   TOUCH_MOTION     +86.62s  0 (0) 33.96/76.30 (272.00/367.00mm)
 event3   TOUCH_FRAME      +86.62s
 event3   TOUCH_MOTION     +86.66s  0 (0) 33.96/74.64 (272.00/359.00mm)
 event3   TOUCH_FRAME      +86.66s
 event3   TOUCH_MOTION     +86.67s  0 (0) 33.96/73.18 (272.00/352.00mm)
 event3   TOUCH_FRAME      +86.67s
 event3   TOUCH_MOTION     +86.69s  0 (0) 34.08/70.27 (273.00/338.00mm)
 event3   TOUCH_FRAME      +86.69s
 event3   TOUCH_MOTION     +86.71s  0 (0) 34.21/68.81 (274.00/331.00mm)
 event3   TOUCH_FRAME      +86.71s
 event3   TOUCH_MOTION     +86.72s  0 (0) 34.33/67.36 (275.00/324.00mm)
 event3   TOUCH_FRAME      +86.72s
 event3   TOUCH_MOTION     +86.74s  0 (0) 34.33/66.74 (275.00/321.00mm)
 event3   TOUCH_FRAME      +86.74s
 event3   TOUCH_MOTION     +86.76s  0 (0) 34.33/66.11 (275.00/318.00mm)
 event3   TOUCH_FRAME      +86.76s
 event3   TOUCH_MOTION     +86.78s  0 (0) 34.46/65.70 (276.00/316.00mm)
 event3   TOUCH_FRAME      +86.78s
 event3   TOUCH_UP         +86.81s
 event3   TOUCH_FRAME      +86.81

[Michael-F-Ellis]

Addendum to prior comment:

I connected a mouse with a scroll wheel (to which the ScrollContainer responds). Libinput reports wheel events from the mouse as POINTER_AXIS events, e.g.

 event5   POINTER_AXIS     +2508.35s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2508.41s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2508.46s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2508.52s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2508.59s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2509.40s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2509.43s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2509.45s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2509.48s    vert -15.00* horiz 0.00 (wheel)
 event5   POINTER_AXIS     +2509.53s    vert -15.00* horiz 0.00 (wheel)

[Michael-F-Ellis]

I have a solution coded, but it's an ugly hack. I'd share the code but it's the sort of thing that really ought only to be sent in a plain brown paper wrapper.

I used goexpect to spawn libinput debug-events and wrote a little state machine to look for touchdown, touchmotion, ..., touchup sequences, extract the y-values, compute deltas against the prior value, and ship the results out on a chan.

In my fyne code, I have a goroutine with a loop that pends on the chan and scales the delta-y values to compute new offsets bounded by the scroller height and the content height. It works and doesn't catch fire but that's about all I can say for it. Obviously, it's completely non-portable.

Oddly enough, all that text parsing is not the performance bottleneck. The loop is running at about 10Hz but the widget lags by at least a half-second to produce visible results. Calling Refresh() seems to be expensive and it behaves like some batching is going on.

FWIW, heres the loop:

    go scrollhack.VScrollReader(e, ychan)
    for {
        dy := <-ychan // will block while no input
        if activePanels != panels.auditLog {
            continue
        }
        // ignore large dy values
        if mu.AbsF64(dy) > 5 { // 5 is empirical
            continue
        }
        // set offset limits
        offSetMin := int64(auditLogReadouts.scroller.Size().Height)
        offSetMax := int64(auditLogReadouts.logText.Size().Height) - offSetMin
        // current offset
        yOff := auditLogReadouts.scroller.Offset.Y
        // compute and constrain new offset
        newYOff := int64(yOff + int(-10*dy)) // 10 is empirical
        newYOff = mu.MinI64(newYOff, offSetMax)
        newYOff = mu.MaxI64(newYOff, offSetMin)
        // Uncomment for debgging
        // info(fmt.Sprintf("dy=%0.2f, newYOff=%d, offSetMin=%d, offsetMax=%d", dy, newYOff, offSetMin, offSetMax))

        // update offset and refresh the widget
        auditLogReadouts.scroller.Offset = fyne.NewPos(0, int(newYOff))
        auditLogReadouts.scroller.Refresh()
    }

[andydotxyz]

Thanks for looking into this @Michael-F-Ellis. I don't know why you sould see a half-second lag, that is strange. The Refresh() call can be expensive - it could relate to where the scroller is in a complex application content - we will be working on optimisations after the 1.3 release as it added some "correctness" at the cost of speed, which we will be able to resolve shortly.

I wonder if there is an opportunity to re-use some of the code in the gomobile dep that we use for the mobile targets. The current issue is that our desktop driver uses GLFW which does not currently support multi-touch. You could see if gomobile may help by running your app in a mobile simualation mode go run -tags mobile . here the drag-to-scroll and other touchscreen style interaction work on my laptop.

This is a complex topic but I am sure we can converge on a good solution.