[Feature was added, convincing community] Mouse interaction?

[Swiftb0y]

Is there the possibility to inject mouse movement like the teensy can do? Or more exactly: Is the Pi able to do this and if it is, are there plans to implement that?

[mame82]

I was thinking about mouse support and the use cases. In fact, I had an exchange on some ideas yesterday.

So to clarify: Emulating a mouse isn't to hard as it is only another kind of HID device, but I'm not going to implement mouse controls like "use the touchscreen of a smartphone to control a PC mouse). If pointer features are going to implemented some day, they'll be controlled from command line.

[Swiftb0y]

Yeah. touchscreen control is to difficult. I would implement it as a terminal function like outhid with screen coordinates and flags for buttons or as a duckyscript extension if that is possible (probably not since its a submodule, right?)

[mame82]

The submodule is written by myself, but I want to keep everything console + SSH based. Additionally I have to test if mouse support consumes additional USB EPs, which I try to avoid. Otherwise not all USB interfaces could be ran concurrently (that's why I disabled USB serial support)

[Swiftb0y]

I think Keyboard and mouse are both part of g_hid. My approach would be to write a simple terminal function that can be used like mousehid x_movement y_movement and mouse click with flags (-1 -2 -3, for left,right and middle mousebutton) for testing purposes. And when you get around to rewrite the duckencoder (on your ToDo.txt), you could implement an extension which allows something like MOUSE x_movement y_movement and MOUSE LEFT/RIGHT/MIDDLE.

[mame82]

I think Keyboard and mouse are both part of g_hid.

Indeed, everything depends on the HID descripors in use. To be precise P4wnP1 uses usb_f_hid instead of g_hid, but EP count depends on different factors.

A standard mouse produces values for relative movement on USB. How many pixels your mouse pointer moves in the end depends on a bumch of factors (driver settings, dpi, acceleration...). So it is not as easy as it seems, although bringing up a HID mouse device could be done in minutes.

[Swiftb0y]

That to technical for me. I just think it would be useful to have this feature. It would be nice to know for the wiki if you are going to implement this. Me and jcstill are working on it: https://github.com/jcstill/P4wnP1/wiki

[mame82]

That's great, guys. Anyway, mouse support shouldn't be promoted as upcoming feature, as long as I haven't got the time to do some experiments with this and applicable uses cases exist

[Swiftb0y]

I think PoisonTap used the mouse to change the default gateway because this cant be done with just the keyboard. And if you have some feedback for the wiki, just open an issue/PR on jcstill's fork.

[mame82]

Now you mix things up. PoisonTap uses DHCP to propagate the default gw. Additionally static routes with 1 bit netmasks are propagated, to spawn two huge private networks covering the IPv4 range (P4wnP1 does sth. similar with the route_spoof option). In case the system has already a default gw, routing decision depends on the targey OS. Windows for example uses the gw with the lowest metric, that's why a ratepatch exists fo P4wnP1

[Swiftb0y]

ohh yeah, it was usbdriveby, not poisontap: https://samy.pl/usbdriveby/ you can see at 1:35 how the DNS settings are changed.

[mame82]

I'm giving it a go https://github.com/mame82/P4wnP1/tree/experimental_mouse

... working on a dedicated script language (very simple).

Mouse could be used as relative pointer (should work on every OS including Android) and absolute pointer (only tested on Windows, does definitely not work on Android).

I have combined the relative and absolute mouse in a single device, which needs some testing when it is done (especially on *nix OS's with X11)

[Swiftb0y]

looks promising so far. I would just update on every command like duckyscript does and add a normal DELAY, but MOVE, MOVETO, CLICK, DOUBLECLICK and BUTTONS for holding and releasing seem logical to me. It would be cool if you could implement it so it can be used from the pi user and add a it to the profile.sh (like I tried in #101 ) so it can be used from the usual ssh session.

[mame82]

Mouse support is introduced with https://github.com/mame82/P4wnP1/commit/c4e6d234628212376336c058dc9d45e5311dce0f

Some feedback would be nice

[mame82]

Documentation is still missing, implemented a small scripting language. Comparison to BashBunny needs to be extended, too.

[Swiftb0y]

I'll experiment with it and add some documentation to the Wiki.

[Swiftb0y]

I just noticed that I won't be able to test it out for the next two weeks, but I still want to add some basic documentation for it. So have you done some testing on OSX and Linux? And another feature that could be useful would be the ability to (optionally) set a screen resolution so you could use absolute percentage values as well as absolute pixel coordinates. I haven't looked at the code yet, but shouldn't that be as easy as extending the parser and adding some extra math to hid_mouse?

[mame82]

I haven't done testing on OSX, but Android and Linux work with relative positioning. According pixel perfect absolute coordinates you're right. This should be added in. The approach of setting screen resolution on beforehand would be similar to teensy