Forward mouse move without button pressed to device

[yume-chan]

• [x] I have checked that a similar feature request does not already exist.

Recently I'm implementing --forward-all-clicks option for my client, and I tried to use ACTION_HOVER_MOVE action for mouse movement events without button pressed. I was surprised that many Apps will react to it

Related code: https://github.com/yume-chan/ya-webadb/blob/36d44243cc1b40dcaa83c77b0b423add5202c8f7/apps/demo/pages/scrcpy.tsx#L676-L681

handlePointerMove = (e: React.PointerEvent<HTMLDivElement>) => {
    this.injectTouch(
        e.buttons === 0 ? AndroidMotionEventAction.HoverMove : AndroidMotionEventAction.Move,
        e
    );
};

(Google Chrome)

(Samsung One UI Settings)

(Google Play Store)

(Twitter, notice second bottom bar button)

It's not very useful, and I'm not actually requesting you to implement it. Just want to let you know it's possible.

[rom1v]

But the mouse is not visible on the device, right? (contrary to scrcpy --hid-mouse)

[yume-chan]

Correct.

I haven't used the HID mouse emulation mode, I know Android will display its own mouse cursor, but do you capture the mouse and hide the cursor on computer? Otherwise there will be two cursors?

In my method injecting hover move, the mouse is still in absolute mode, so hover state on Android Apps is always under the computer mouse cursor, and Android won't display its own cursor.

[rom1v]

but do you capture the mouse and hide the cursor on computer?

Yes.

In my method injecting hover move, the mouse is still in absolute mode, so hover state on Android Apps is always under the computer mouse cursor, and Android won't display its own cursor.

OK, I will try, and maybe enable it or add an option.

[hammad2224]

just like shooting games if we could do the mouse move like appPower mirror. we can play alot of games with this software. Can you please implement this feature i am waiting from last 2 years for this feature so i could play shooting games on it.

Also please let us know how to donate to your project

[rom1v]

and I tried to use ACTION_HOVER_MOVE action for mouse movement events without button pressed. I was surprised that many Apps will react to it

I made a quick test:

diff --git a/app/src/mouse_inject.c b/app/src/mouse_inject.c
index 2e89de9a..1d5a975b 100644
--- a/app/src/mouse_inject.c
+++ b/app/src/mouse_inject.c
@@ -58,17 +58,13 @@ convert_touch_action(enum sc_touch_action action) {
 static void
 sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
                                     const struct sc_mouse_motion_event *event) {
-    if (!event->buttons_state) {
-        // Do not send motion events when no click is pressed
-        return;
-    }
-
     struct sc_mouse_inject *mi = DOWNCAST(mp);

     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
         .inject_touch_event = {
-            .action = AMOTION_EVENT_ACTION_MOVE,
+            .action = event->buttons_state ? AMOTION_EVENT_ACTION_MOVE
+                                           : AMOTION_EVENT_ACTION_HOVER_MOVE,
             .pointer_id = POINTER_ID_MOUSE,
             .position = event->position,
             .pressure = 1.f,

On a Nexus 5, it has no effect. On a OnePlus 7 Pro, it only "works" in the notification panel, but not in settings or context menus…

Maybe some ACTION_HOVER_ENTER and ACTION_HOVER_EXIT are necessary?

just like shooting games if we could do the mouse move like appPower mirror. we can play alot of games with this software.

How does APowerMirror send mouse event? (I've never used it)

[hammad2224]

i think they use the swap and merge it with mouse move. like key mapping along with mouse mapping here is the github repo which also allow mouse swap and multiple key combinations. Basically we need key mapping functionality for gaming experience like LD Player Smartgaga https://github.com/BobbleKeyboard/android_touch

[yume-chan]

On a Nexus 5, it has no effect. On a OnePlus 7 Pro, it only "works" in the notification panel, but not in settings or context menus…

Yes, the ultimate outcome is decided by each App.

Maybe some ACTION_HOVER_ENTER and ACTION_HOVER_EXIT are necessary?

I didn't try them. I found ACTION_HOVER_MOVE works also by accident. Maybe more Apps will work when having them.

[yume-chan]

Samsung S Pen supports hovering, so Samsung One UI might be the best system in supporting hover state.

I did more tests around ACTION_HOVER_ENTER and ACTION_HOVER_EXIT, looks like no one cares about them.

One thing I found is that when another pointer is DOWN, the hovering pointer must be removed, otherwise the new pointer won't do anything. With a real OTG mouse, touching the screen with finger will automatically deactivate the mouse.

[AndroidDeveloperLB]

Any news about this?

On Android Studio, it's implemented right now, so when you hover with your mouse over (probably almost and not all) anything , it makes an effect for it accordingly.

[rom1v]

I just re-tested this patch: https://github.com/Genymobile/scrcpy/issues/3070#issuecomment-1060047511

It seems to work on my Pixel 8. Could you please test?

If that works, we can add an option, either:

• --forward-hover to enable the feature
• enable hover by default, and add --no-forward-hover to restore the previous behavior

(other suggestions for naming are welcome)

[AndroidDeveloperLB]

@rom1v I can test on Pixel 6 with Android 14 and Honor Magic 4 pro with Android 14. If you want me to check, please provide me a link to the ScrCpy binaries that have this feature.

[rom1v]

@AndroidDeveloperLB Here is a binary with the patch above:

• scrcpy-win64-hover.zip SHA-256: 9b3851b888e9dce4759e3c753b0445e46e48de91a8435fa32680b6dde9576cc

[AndroidDeveloperLB]

@rom1v Thank you. Works fine for Honor Magic4 pro (though for settings they probably didn't add hovering effect for some reason). Also works fine for Pixel 6.

Now I wonder if you will make it the default behavior or not, and if it has any danger in using it... Is it similar to what Google did? If so, maybe it's safe...

[rom1v]

Implemented by #5039.

[AndroidDeveloperLB]

@rom1v Thank you! I hope some day Google will pay you a lot for this tool, and users will also have it official, like what Apple is doing today with their mirroring solution.