search

Genymobile / scrcpy

Display and control your Android device
Apache License 2.0
113.39k stars 10.8k forks source link

Unable to control Scrcpy because of the absence of mouse cursor's functionality, when interacting with experimental desktop from android developer settings #4829

Open onyhen opened 7 months ago

onyhen commented 7 months ago

Environment

Describe the bug A clear and concise description of what the bug is.

On errors, please provide the output of the console (and adb logcat if relevant).

Please paste terminal output in a code block.

Please do not post screenshots of your terminal, just post the content as text instead.

onyhen commented 7 months ago

Hello & Greetings, Genymobile. The only reason why I am asking questions is because I have a negative issue. To start with, I explored about Android's Developer connectivity, it's Debug Bridge. And I use Windows 11. I wanted to use the forced experimental desktop from an Android phone to operate on a computer. So I used your well-known & trusted program, Scrcpy with ADB, to link both wired and wirelessly. Successfully, I accessed the experimental destop. :-) , But I cannot control it with my mouse cursor! <:-( . I was only able to operate with the keyboard as normal, but sadly, I was not able to do almost anything without the cursor control. I would thankfully appreciate the instructions that you provide me. Thanks.

onyhen commented 7 months ago

By the way, even though I had to install some more applications and had to type in some terminal code for experimental desktop to work, Scrcpy still runs in it's normal mirroring technology. But I mainly need help to fix my mouse console that refuses to work. Thanks :-)

rom1v commented 7 months ago

What if you pass --mouse=uhid? https://github.com/Genymobile/scrcpy/blob/master/doc/mouse.md#uhid

onyhen commented 7 months ago

I entered it in, but the cursor was in the phone's screen instead of the computer. Like this: The cursor was trapped in the phone screen, but I wasn't able to control the forced experimental desktop. Sorry to say.

rom1v commented 7 months ago

but the cursor was in the phone's screen instead of the computer.

Yes, I thought this was what you wanted.

If you don't do that, then the cursor "stays" on the computer and you can click. So what do you want to achieve?

onyhen commented 7 months ago

I wasn't using screen mirroring, but was using the 'Experimental Desktop' mode on 'Android Developer settings'. So the thing is that the phone sends out it's desktop view of itself on another screen, while it's on something else.

image

rom1v commented 7 months ago

but was using the 'Experimental Desktop' mode on 'Android Developer settings'.

Oh, ok. The UHID keyboard behaves (or should behave) as if you plugged a physical keyboard on your device. What happens if you plug/connect a physical keyboard?

I don't know if the UHID mouse can be configured to work on a separate virtual screen.

onyhen commented 7 months ago

Wow. Thank you so much for helping me out! :-)

Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

From: Romain Vimont @.> Sent: Tuesday, April 9, 2024 4:42:14 PM To: Genymobile/scrcpy @.> Cc: onyhen @.>; Author @.> Subject: Re: [Genymobile/scrcpy] Unable to control Scrcpy because of the absence of mouse cursor's functionality, when interacting with experimental desktop from android developer settings (Issue #4829)

but was using the 'Experimental Desktop' mode on 'Android Developer settings'.

Oh, ok. The UHID keyboard behaves (or should behave) as if you plugged a physical keyboard on your device. What happens if you plug/connect a physical keyboard?

I don't know if the UHID mouse can be configured to work on a separate virtual screen.

— Reply to this email directly, view it on GitHubhttps://github.com/Genymobile/scrcpy/issues/4829#issuecomment-2045514200, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHIM66SO7YSFZBKHP4AQ6ELY4QD5NAVCNFSM6AAAAABF5HKRVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBVGUYTIMRQGA. You are receiving this because you authored the thread.Message ID: @.***>

yume-chan commented 7 months ago

I don't know if the UHID mouse can be configured to work on a separate virtual screen.

I think it's impossible. Presume the "official" virtual mouses work on virtual displays, it uses InputManagerGlobal.addUniqueIdAssociation and InputManagerInternal.setVirtualMousePointerDisplayId

https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/services/companion/java/com/android/server/companion/virtual/InputController.java;l=304;drc=3d19fbcc09b1b44928639b06cd0b88f735cd988d

https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/services/companion/java/com/android/server/companion/virtual/InputController.java;l=162;drc=3d19fbcc09b1b44928639b06cd0b88f735cd988d

I tried InputManagerGlobal.addUniqueIdAssociation, it only causes our UHID mouse to stop functioning. InputManagerInternal.setVirtualMousePointerDisplayId is not exposed.

Anyway, Android only has one mouse cursor so making it work on multiple displays is also impossible.

rom1v commented 7 months ago

Thank you for your investigations.

@onyhen What's the problem with the normal mouse mode (when you don't pass --mouse=hid)? You should still be able to click, can't you?