How to implement mouse HID swipe/drag emulation (press-move-release)?

[bluepuma77]

Just managed to implement a mouse emulation for iOS with relative and absolute mouse coordinates, thanks to the support on the other issue here (link). Next challenge: Is it possible to implement a mouse swipe (often needed to delete items) ?

I tried with this code, but it is always interpreted as a mouse click, not a swipe:

    m.move(100, 0)
    m.press(Mouse.LEFT_BUTTON)
    for i in range(1, 20):
        m.move(-5, 0)
        time.sleep(0.1)
    m.release(Mouse.LEFT_BUTTON)

I wonder if it has something to do or is similar to this issue, if those mouse move commands would need to send some mouse pressed info with them, which might not be implemented.

Any idea how to do a swipe correctly?

[dhalbert]

I looked at the code and I think swipe (or I would say "drag") should be working. The press/release methods modify self.report[0], which is where the button info is kept. The move() method does not disturb self.report[0], so the current state of the buttons should be sent in the report sent by move(). So I'm not sure why you're not seeing a drag operation. I'll have to try it out. What is currently on the screen when you try the swipe/drag, and what is the OS?

In the case of AbsoluteMouse, the coordinates are absolute, so _send_no_move() has to send the current position in the report, instead of zeros.

[bluepuma77]

what is the OS

I am trying to control an iOS tablet (iPad), want to delete a lot of items, which require a drag (swipe) to show delete button.

What is currently on the screen

I am on the screen with list items (think emails), and then need to be swiped to the left to be able to click delete.

Instead of dragging, the iOS is interpreting as mouse move and the release as a simple click, showing email details.

[dhalbert]

I tested your exact example above on a regular desktop computer, and it does successfully do dragging. Here's an example, where it selected some text by dragging: So I think there is something about what the iPad is expecting in terms of drag speed or distance that is not matching what your code is sending. On my iPhone, for instance, if I try to drag left slowly in an email inbox list, I don't get the "Delete", etc. items showing up: instead the list item itself just moves slightly and then moves back on button up.

You might try moving more quickly, moving further, moving in finer granularities, etc. There might be some accessibility settings that would help. I tried doing some websearches about advice, but didn't find anything immediately.

[bluepuma77]

@dhalbert Thanks for your extensive support. I just started from scratch with a clean code base:

import time
import usb_hid
from adafruit_hid.mouse import Mouse

# Set up a mouse device.
m = Mouse(usb_hid.devices)

print("Go home (0, 0)")
for i in range(1, 6):
    m.move(-50, -50, 0)
    time.sleep(0.8)

print("Go (140, 140)")
m.move(140, 140, 0)
time.sleep(1)

print("Press")
m.press(Mouse.LEFT_BUTTON)
time.sleep(0.1)

for i in range(1, 50):
    m.move(-3, 0, 0)
    time.sleep(0.02)

print("Release")
m.release(Mouse.LEFT_BUTTON)
time.sleep(1)

The iPad interprets the action as simple click, not drag/swipe.

---

I then connected a real gamer USB mouse, turns out the clicks are not recognized at all by the iPad, move is okay. Being curious, I then connected a standard USB mouse, move is okay, click is recognized, but drag is only recognized as click.

So it seems to be an issue with the iPad HID interpretation, which varies even by device from the same class.

---

A chatbot recommends to use iOS "Assistive Touch Settings" to implement a work-around:

Assistive Touch Settings: Go to Settings > Accessibility > Touch > AssistiveTouch and enable it. This setting allows you to create custom gestures (like swipes or drags) and assign them to mouse buttons. For example, you could create a left-to-right swipe and then trigger it through a secondary click or a long press.

As it seems to be iOS-specific, I will close this for now.