ELAN1200 Touchpad: pointer does not move smoothly, creates phantom clicks, and generates log errors

[qwrd]

Qubes OS version Qubes R4.0.2-rc2

Affected component(s) or functionality Elan 1200 Touchpad

Brief summary The touchpad doesn't work properly.

To Reproduce

1. Use a laptop with ELAN1200 touchpad (I'm using Zenbook UX410UA)
2. Install Qubes R4.0.2-rc2 on it, optionally update dom0
3. Try to use the touchpad to move the cursor

Expected behavior The touchpad should work without any errors or erratic behavior.

Actual behavior The touchpad initially works for a few seconds then it gradually breaks down completely. Mouse pointer doesn't move smoothly, phantom clicks are created, and the syslog gets spammed with errors.

Screenshots N/A

Additional context Okay, so my laptop won't let me touch its naughty place. Each time I try to use the touchpad it will initially work for a few seconds, but then the behavior will quickly worsen with continued use until it becomes completely unusable (I'm talking about like 10 seconds or so after first use). The symptoms include jumping cursor (dropped frames), phantom left button clicks, random scrolls, click & drag events as the text gets selected, etc. All this happens when using a single finger to move the cursor without pressing any buttons or using the mouse.

When happening, the syslog will get spammed with messages like these:

[  317.989978] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  317.990656] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  318.046554] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  318.047136] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  318.102763] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  318.103339] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  318.168683] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/82)
[  319.705539] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/82)
[  319.762092] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/82)
[  320.050799] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  320.107179] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)
[  320.107769] i2c_hid i2c-ELAN1200:00: i2c_hid_get_input: incomplete report (16/16466)

Notice that there are two types of errors. The 16/82 lines only rarely happen when I am touching the touchpad with a single finger and moving it around a bit. But as soon as I remove my finger, the 16/16466 lines will be printed out with very high frequency until I put my finger back or until some time later after a very large number of them are printed out - I'm assuming this is when the touchpad goes to sleep due to inactivity. I've done some searching and it seems like these bugs should already be patched some recent kernels. As usual, I've tested the hardware in Debian Buster 10 Live LXQt and Ubuntu Mate 19.10 Live, and the touchpad appears to work fine in both cases. Doing a "rmmod i2c-hid" stops the syslog errors, but also disables the touchpad.

Some more info:

[root@dom0 ~]# dmesg | grep ELAN
[   17.507865] i2c_hid i2c-ELAN1200:00: i2c-ELAN1200:00 supply vdd not found, using dummy regulator
[   17.507875] i2c_hid i2c-ELAN1200:00: Linked as a consumer to regulator.0
[   17.507876] i2c_hid i2c-ELAN1200:00: i2c-ELAN1200:00 supply vddl not found, using dummy regulator
[   17.539311] input: ELAN1200:00 04F3:3044 Mouse as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-6/i2c-ELAN1200:00/0018:04F3:3044.0001/input/input7
[   17.539365] input: ELAN1200:00 04F3:3044 Touchpad as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-6/i2c-ELAN1200:00/0018:04F3:3044.0001/input/input8
[   17.539412] hid-generic 0018:04F3:3044.0001: input,hidraw0: I2C HID v1.00 Mouse [ELAN1200:00 04F3:3044] on i2c-ELAN1200:00
[   17.742568] input: ELAN1200:00 04F3:3044 Touchpad as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-6/i2c-ELAN1200:00/0018:04F3:3044.0001/input/input18
[   17.742703] hid-multitouch 0018:04F3:3044.0001: input,hidraw0: I2C HID v1.00 Mouse [ELAN1200:00 04F3:3044] on i2c-ELAN1200:00

Output for dmesg is almost identical in debian/ubuntu live so I won't post those.

[root@dom0 ~]# lspci
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 02)
00:02.0 VGA compatible controller: Intel Corporation HD Graphics 620 (rev 02)
00:04.0 Signal processing controller: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem (rev 02)
00:14.0 USB controller: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller (rev 21)
00:14.2 Signal processing controller: Intel Corporation Sunrise Point-LP Thermal subsystem (rev 21)
00:15.0 Signal processing controller: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #0 (rev 21)
00:15.1 Signal processing controller: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #1 (rev 21)
00:16.0 Communication controller: Intel Corporation Sunrise Point-LP CSME HECI #1 (rev 21)
00:17.0 SATA controller: Intel Corporation Sunrise Point-LP SATA Controller [AHCI mode] (rev 21)
00:1c.0 PCI bridge: Intel Corporation Sunrise Point-LP PCI Express Root Port #1 (rev f1)
00:1c.7 PCI bridge: Intel Corporation Sunrise Point-LP PCI Express Root Port #8 (rev f1)
00:1f.0 ISA bridge: Intel Corporation Device 9d4e (rev 21)
00:1f.2 Memory controller: Intel Corporation Sunrise Point-LP PMC (rev 21)
00:1f.3 Audio device: Intel Corporation Sunrise Point-LP HD Audio (rev 21)
00:1f.4 SMBus: Intel Corporation Sunrise Point-LP SMBus (rev 21)
02:00.0 Network controller: Intel Corporation Wireless 8260 (rev 3a)

[root@dom0 ~]# libinput-list-devices
...
Device:           ELAN1200:00 04F3:3044 Touchpad
Kernel:           /dev/input/event8
Group:            6
Seat:             seat0, default
Size:             103.23x70.90mm
Capabilities:     pointer 
Tap-to-click:     disabled
Tap-and-drag:     enabled
Tap drag lock:    disabled
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: disabled
Calibration:      n/a
Scroll methods:   *two-finger edge 
Click methods:    *button-areas clickfinger 
Disable-w-typing: enabled
Accel profiles:   none
Rotation:         n/a
...

Solutions you've tried

1. Upgraded the kernel (4.19.82-1.pvops.qubes.x86_64) to kernel-latest (5.3.7-1.pvops.qubes.x86_64 or 5.3.9-1.qubes.x86_64) in dom0 - no dice

Relevant documentation you've consulted https://www.qubes-os.org/doc/newer-hardware-troubleshooting/

Related, non-duplicate issues none

[sorushsaghari]

i have this problem of touchpad not working on kernel 5.4 or higher

[ThaSami]

same problem here on asusfx503vm Fedora31 kernel 5.4 +

[qwrd]

This issue is still present in Qubes 4.1 Build20200310-4.1

[0spinboson]

at least one of the two messages looks rather similar to https://patchwork.kernel.org/patch/10869095/ , and if you search the web, there are multiple results describing problems with these multitouch touchpads. I don't think this is a qubes-related issue.

[andrewdavidwong]

@marmarek, can you confirm whether this is not our bug?

[m-v-b]

@andrewdavidwong , I have a laptop with ELAN1200 touchpad as well, and I can confirm that this is not a bug in Qubes OS, but rather caused by Xen.

Xen acknowledges an interrupt if a virtual machine (for example, dom0) does not acknowledge the interrupt within 1 millisecond.

I have patched Xen and the i2c-hid driver to locally resolve this issue as follows, but in my opinion these changes are not acceptable for merging by upstream or by Qubes OS due to their nature:

1. Xen: Allow customizing the IRQ servicing time-out via the Xen command line. (I have set the time-out to 40 milliseconds.)
2. Xen: Stop the corresponding IRQ servicing timer when a VM acknowledges an interrupt.
3. Xen: Allow the user to override the trigger type (i.e., positive edge-triggered vs high level-triggered) of an interrupt via the command line. This should ideally not be necessary, as the i2c-hid standard mandates high level-triggered interrupts rather than positive edge-triggered interrupts, but my system behaves better with positive edge trigger type.
4. Linux: i2c-hid: Silence the logs in question. This change should not be necessary either, but the kernel continues to print out a single line every time I lift my finger from the touchpad, so I had to silence the error logs.

After applying the aforementioned modifications, if I enable the debugging output for this touchpad's device driver via sudo modprobe -r i2c_hid; sudo modprobe -i -v i2c_hid debug=1, and if I use the touchpad for a few seconds, I can observe that successive interrupt service routine invocations have about 16 milliseconds in between:

[ 3813.395507 <    0.016334>] i2c_hid i2c-ELAN1200:00: input: 10 00 04 03 9d 09 ae 02 1a 60 01 00 22 54 00 00
[ 3813.411770 <    0.016263>] i2c_hid i2c-ELAN1200:00: input: 10 00 04 03 69 09 b2 02 c4 60 01 00 20 55 00 00
[ 3813.428093 <    0.016323>] i2c_hid i2c-ELAN1200:00: input: 10 00 04 03 40 09 b4 02 64 61 01 00 20 55 00 00

All this to say, the touchpad's interrupt requests are acknowledged too early by Xen, which appears to confuse the touchpad device.

[andrewdavidwong]

Thanks for the detailed reply, @m-v-b

In light of this, I'm closing this issue as "not our bug." If anyone believes this is a mistake, please leave a comment, and we'll be happy to take another look. Thank you.

[dylangerdaly]

Can we re-open this issue? A bunch of devices use ELAN touchpads, and this appears to be affecting me.

[andrewdavidwong]

@dylangerdaly: Reopened, but do you have reason to believe that this is actually a Qubes OS bug (i.e., the Qubes OS Project actually develops the software that is causing this bug)? Based on the previous comments on this issue, it looks like it's not. (Related FAQ: Why don’t you fix upstream bugs that affect Qubes OS?)

Also, which Qubes release are you on?

[dylangerdaly]

It's likely not specially a QubesOS bug, but it's affecting me, I'll try to log this with the xen-mailing list, it's the only ticket in existence so it's good to have it open.

I'm on R4.2RC3

Also my issue is a little different, I don't get any input at all, I'm running the ELAN07A8:00

[andrewdavidwong]

It's likely not specially a QubesOS bug, but it's affecting me, I'll try to log this with the xen-mailing list, it's the only ticket in existence so it's good to have it open.

It is our policy not to keep issues open that are not actionable for the Qubes developers. Besides, issues still remain in existence and are fully discoverable even when they are closed.

Reclosing as "not our bug" (the Qubes OS Project does not develop the software that is causing this bug). If anyone has another reason for reopening this issue, please leave a comment, and we'll be happy to take another look. Thank you.

[andrewdavidwong]

Just added this section:

https://www.qubes-os.org/doc/issue-tracking/#understanding-open-and-closed-issues

Hope it helps to clarify the situation on open vs. closed issues.

[DemiMarie]

Are the patches that @m-v-b used available anywhere?

[dylangerdaly]

Ooooo would be great to get this solved

[DemiMarie]

Reopening as this needs to be reported to xen-devel and possibly fixed via a downstream backport.

[m-v-b]

Are the patches that @m-v-b used available anywhere?

Hi @DemiMarie,

The patches I used were written by myself, and I did not publish them at the time. @euidzero exchanged a few e-mails with me and a software engineer at AMD regarding Qubes OS issue #8734, and in that e-mail thread I shared a copy of the (first) patch that allows customization of the interrupt servicing timeout (the default for which is 1 millisecond) with a Xen command line argument.

My second patch related to cancelling the interrupt servicing timer is no longer needed, thanks to the following commit in Xen:

  commit 359cf6f8a0ec ("x86/IRQ: don't keep EOI timer running without need")
  (4.13.0-rc1~819 )

I did not think that my third patch (which allows customizing interrupt trigger modes such as positive edge vs. high level) would be necessary, and given the ugly debugging nature of the patch, I would rather keep it private for now, but if this patch would be helpful, please let me know.

Finally, all of my patches were written against Xen 4.8.5, so they will require porting to recent Xen versions, but this should be relatively straightforward to carry out.

I would be happy to publish the first patch and link to it from here.

Thank you,

Vefa

[dylangerdaly]

Just to confirm, do these patches totally fix ELAN Touchpad devices for Qubes on Ryzen 7000?

[m-v-b]

Hi @dylangerdaly,

The first Xen patch in question (that I had written to allow customizing the interrupt acknowledgement timeout from its default of 1 millisecond, with a Xen command line argument) was sufficient for me in resolving the flood of the error messages that read i2c_hid_get_input: incomplete report, and the remaining patches were mainly intended as enhancements.

Despite this, I unfortunately cannot guarantee that the same patch will resolve your issues, because my laptop had an Intel CPU, and I am not sure if there are other variables/unknowns.