Touchscreen support meta-issue

[ids1024]

Cosmic-comp has support for wl_touch, passing through touchscreen events from libinput to clients that handle wl_touch.

That's the main thing that's needed to make a touchscreen on a laptop somewhat useful, but there are more things that are needed for full touchscreen support. Especially for use of tablets without keyboards.

• On-screen keyboard
• Touch support in iced-sctk
• Touch support in xdg-shell-wrapper/cosmic-panel
• Touch support for SSD and other UI in cosmic-comp
  • Started in https://github.com/pop-os/cosmic-comp/pull/386
  • Moving and resizing windows with touchscreen

First-class support for touchscreen-only devices isn't the highest priority, but it's something to consider.

[Quackdoc]

For touchscreens; It would be nice to have cosmic-settings expose touchscreen settings like display mapping. Currently on my desktop my touchscreen is unusable since it isn't mapped, IIRC I saw some settings for it in cosmic-comp? but i'm not sure they are exposed anywhere.

For OSKs; It would be nice if it's a more... Flexible solution. I'm not really sure how OSKs are handled on wayland in general. though I thought it was just the text-input protocol and then up to the compositor but I often find references to QT_IM modules and what not

However the current status quo is a massive pain point for a lot of people, myself included. OSKs are often require platform specific invocation despite actually being relatively flexible. To give examples, Phosh relies on their OSK0 dbus interface and requires users to use a bridge/shim application to get other keyboards working. Plasma only really works with mallit/mallit only really working with plasma etc. wf-osk is pretty basic and im not even sure if wayfire uses it or not, same with wvkbd.

Meanwhile actually triggering and dismissing OSKs are generally easy, Squeekboard can be triggered and dismissed with busctl call --user sm.puri.OSK0 /sm/puri/OSK0 sm.puri.OSK0 SetVisible b {true,false} wvkbd can be triggered and dismissed with kill -{USR1,USR2} $(pgrep -x "wvkbd-mobintl") wf-osk I believe you just trigger it and dismiss it by killing it and starting it.

Being able to choose an appropriate OSK would be a really nice feature that doesn't really work on other compositors. Personally I really like wvkbd and find squeeboard really hard to use, but I know many people prefer maliit, squeekboard etc.

I've tested both squeekboard and wvkbd. The OSKs I've tested were dismissed for some reason when I tried to use them with firefox. (though wvkbd worked when I retriggered it) However they both worked when interacting with cosmic-term and telegram. wvkbd-mobintl -L 200 (-L is landscape height) currently for testing.

As for manually showing and dismissing keyboards that can easily be done using an applet.

[ids1024]

Touch input was added to iced-sctk in https://github.com/pop-os/iced/pull/135.

For cosmic-panel, this will require https://github.com/pop-os/xdg-shell-wrapper/pull/22 and https://github.com/pop-os/cosmic-panel/pull/199.

[ahoneybun]

From testing the cosmic-panel PR and I think that pulls in the xdg-shell-wrapper one the only issues so far are:

• Able to move COSMIC applications (I can move GTK and others either way).
• Using the toggles in Applets (you can trigger certain ones like opening Settings from the User Applet).
• Dialogs and their buttons can not be pressed.

[ninjaquick]

Touch inputs do not map to the correct display (I can create a new item if this is something that's expected to be working):

• Display: PLANAR PCT2785
• Touch Device (lsusb): ID: 2575:0402 Weida Hi-Tech Co., LTD CooTouchR System

  Bus 001 Device 018: ID 2575:0402 Weida Hi-Tech Co., Ltd. CoolTouchR System           
  Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 [unknown]
  bDeviceSubClass         0 [unknown]
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x2575 Weida Hi-Tech Co., Ltd.
  idProduct          0x0402 CoolTouchR System           
  bcdDevice           13.30
  iManufacturer           1 Weida Hi-Tech               
  iProduct                2 CoolTouchR System           
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
  bLength                 9
  bDescriptorType         2
  wTotalLength       0x0022
  bNumInterfaces          1
  bConfigurationValue     1
  iConfiguration          3 CoolTouchR System 10x       
  bmAttributes         0xa0
    (Bus Powered)
    Remote Wakeup
  MaxPower              500mA
  Interface Descriptor:
    bLength                 9
    bDescriptorType         4
    bInterfaceNumber        0
    bAlternateSetting       0
    bNumEndpoints           1
    bInterfaceClass         3 Human Interface Device
    bInterfaceSubClass      0 [unknown]
    bInterfaceProtocol      0 
    iInterface              4 CoolTouchR USB Interface 1  
      HID Device Descriptor:
        bLength                 9
        bDescriptorType        33
        bcdHID               1.10
        bCountryCode            0 Not supported
        bNumDescriptors         1
        bDescriptorType        34 Report
        wDescriptorLength     747
        Report Descriptors: 
          ** UNAVAILABLE **
    Endpoint Descriptor:
      bLength                 7
      bDescriptorType         5
      bEndpointAddress     0x81  EP 1 IN
      bmAttributes            3
        Transfer Type            Interrupt
        Synch Type               None
        Usage Type               Data
      wMaxPacketSize     0x0040  1x 64 bytes
      bInterval               1
  Device Status:     0x0000
  (Bus Powered)

  Inputs are detected but map to coordinates on the primary display.

Behaviors to compare to:

• Pop!OS_ 22.04: Input is mapped to the correct display automatically.
• VanillaOS 2 (Debian): Input is mapped to the correct display automatically.
• KDE: Inputs don't appear to map automatically, but can be assigned via the Settings app.

Use cases to consider this:

• LCD-backed trackpads / "Apple bar".
• POS/Video Walls
• Disability assistance devices
• Medical, mechanical or other environments where multi-display, multi-input with one computer may exist
• "GameGlass" like native implementations
• Media screens / Secondary displays which do not run content which needs mouse/keyboard input frequently

My personal use-case is I have a secondary system at my desk which is a laptop and the above 27" monitor. I use is as a more natural input for webapps, multimedia, discord calling/other calling apps, some tinkering to make more flexible and interesting gameglass-like solutions.

I am not sure about wacom, if it natively combines the taps, but this could be something that needs to be handled if wacom/drawing tablets require this type of care. I don't have a device of this kind to test with.

[Vixeliz]

I don't know if this would be out of the scope and stay seperate (ie like lisgd). But having remappable touchscreen gestures would also be very cool

[Quackdoc]

@ninjaquick missed this a while ago, you can bind to specific display.

Get device name: libinput list-devices | grep Device\: ... Device: ILITEK ILITEK-TP ...

.config/cosmic/com.system76.CosmicComp/v1/input_devices
----------------------------
{
 "ILITEK ILITEK-TP": (
    state: Enabled,
    map_to_output: Some("HDMI-A-2")
    ),
}