Add scrcpy window without video playback

rom1v commented 3 weeks ago

Add the possibility to solely control the device without screen mirroring:

scrcpy --no-video --no-audio

This is different from OTG mode, which does not require USB debugging at all. Here, it is just the standard mode but with the possibility to disable video playback.

By default, always open a window (even without video playback), and add an option --no-window.

This impacts the behavior of some usages.

For example, the following command used to only play audio without video or controls:

scrcpy --no-video

Now, it opens a window, and allows to control it using the keyboard and mouse (by default, mouse mode is switched to UHID if video mirroring is disabled, because a relative mouse mode is required).

window

The controls must be disabled explicitly if necessary to play audio only:

scrcpy --no-video --no-control

To get the same behavior as before (play audio without window), disable the window:

scrcpy --no-window

(this implicitly set --no-control and --no-video-playback, which in turn set --no-video if there is no recording or V4L2 sink)

Fixes #4727 Fixes #4793

Here is a binary for Windows:

scrcpy-pr4868.4.zip _{SHA-256: b769c9bc403d91d78fae3efa87856dbe2af5905c8d3a33392beaf5fa7103c30}

old

- [`scrcpy-pr4868.zip`](https://tmp.rom1v.com/scrcpy/4868/1/scrcpy-pr4868.zip) _{`SHA-256: cf100b210e75c19d58f50b3c704d193a6ddbd45520928b7f83acc40a8246815`}

AnshulJ999 commented 2 weeks ago

-KM --no-video --no-audio -d scrcpy 2.4 https://github.com/Genymobile/scrcpy INFO: ADB device found: INFO: --> (usb) JZU4CURGLJA6P74H device Realme_Pad_LTE INFO: (tcpip) 192.168.1.204:5555 device AFTSSS C:\scrcpy\scrcpy-server: 1 file pushed, 0 skipped. 23.1 MB/s (69243 bytes in 0.003s) [server] INFO: Device: [Realme] Realme Realme Pad LTE (Android 13) INFO: Renderer: directed

Hey, so this is what I got when using the command you mentioned on Reddit. It seems the command works, and there's a window with an Android logo (same as OTG mode), but it quickly disappears and exits the command.

Also I'm not sure if this is possible yet via the --no-window or --no-video option, but just wanted to add this feature request from my Reddit comment:

A windowless approach to OTG mode (meaning it wouldn't be a window, it could exist in the system tray maybe), where a user-defined hotkey can be used to toggle between the host screen and the android device. This would make it very similar to deskdock.

I believe the rest of the things work already: ADB wifi works, UHID keyboards have gestures and hotkeys (unlike OTG mode).

Thanks.

rom1v commented 2 weeks ago

Thank you for your report.

but it quickly disappears and exits the command.

Without any error message?

I cannot reproduce the problem. Maybe because the binary was from an old version of this PR. Please test this new one:

scrcpy-pr4868.2.zip _{SHA-256: 2d0827e876485d5afb5476577953efc2d3df7fb81ee6e30092f5d94b41b125c}

Also, do you have the same issue with another device?

A windowless approach to OTG mode (meaning it wouldn't be a window, it could exist in the system tray maybe), where a user-defined hotkey can be used to toggle between the host screen and the android device. This would make it very similar to deskdock.

Technically, scrcpy receives events from the scrcpy window. If there is no window, I don't think it is possible to receive the events with SDL. It would only be possible by platform-specific means I guess.

rom1v commented 2 weeks ago

I cannot reproduce the problem.

OK, on Windows, I can reproduce. I will investigate.

rom1v commented 2 weeks ago

OK, with these two changes it should be fixed:

fix 1

```diff diff --git a/app/src/screen.c b/app/src/screen.c index cda562468..9ee61383c 100644 --- a/app/src/screen.c +++ b/app/src/screen.c @@ -272,6 +272,8 @@ sc_screen_render(struct sc_screen *screen, bool update_content_rect) { static int event_watcher(void *data, SDL_Event *event) { struct sc_screen *screen = data; + assert(screen->video); + if (event->type == SDL_WINDOWEVENT && event->window.event == SDL_WINDOWEVENT_RESIZED) { // In practice, it seems to always be called from the same thread in @@ -477,7 +479,9 @@ sc_screen_init(struct sc_screen *screen, sc_input_manager_init(&screen->im, &im_params); #ifdef CONTINUOUS_RESIZING_WORKAROUND - SDL_AddEventWatch(event_watcher, screen); + if (screen->video) { + SDL_AddEventWatch(event_watcher, screen); + } #endif static const struct sc_frame_sink_ops ops = { ```

fix 2

```diff diff --git a/app/src/input_manager.c b/app/src/input_manager.c index cd2706559..5154d474e 100644 --- a/app/src/input_manager.c +++ b/app/src/input_manager.c @@ -626,6 +626,23 @@ sc_input_manager_process_key(struct sc_input_manager *im, im->kp->ops->process_key(im->kp, &evt, ack_to_wait); } +static struct sc_position +sc_input_manager_get_position(struct sc_input_manager *im, int32_t x, + int32_t y) { + if (im->mp->relative_mode) { + // No absolute position + return (struct sc_position) { + .screen_size = {0, 0}, + .point = {0, 0}, + }; + } + + return (struct sc_position) { + .screen_size = im->screen->frame_size, + .point = sc_screen_convert_window_to_frame_coords(im->screen, x, y), + }; +} + static void sc_input_manager_process_mouse_motion(struct sc_input_manager *im, const SDL_MouseMotionEvent *event) { @@ -635,12 +652,7 @@ sc_input_manager_process_mouse_motion(struct sc_input_manager *im, } struct sc_mouse_motion_event evt = { - .position = { - .screen_size = im->screen->frame_size, - .point = sc_screen_convert_window_to_frame_coords(im->screen, - event->x, - event->y), - }, + .position = sc_input_manager_get_position(im, event->x, event->y), .pointer_id = im->forward_all_clicks ? POINTER_ID_MOUSE : POINTER_ID_GENERIC_FINGER, .xrel = event->xrel, @@ -760,12 +772,7 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im, uint32_t sdl_buttons_state = SDL_GetMouseState(NULL, NULL); struct sc_mouse_click_event evt = { - .position = { - .screen_size = im->screen->frame_size, - .point = sc_screen_convert_window_to_frame_coords(im->screen, - event->x, - event->y), - }, + .position = sc_input_manager_get_position(im, event->x, event->y), .action = sc_action_from_sdl_mousebutton_type(event->type), .button = sc_mouse_button_from_sdl(event->button), .pointer_id = im->forward_all_clicks ? POINTER_ID_MOUSE @@ -840,11 +847,7 @@ sc_input_manager_process_mouse_wheel(struct sc_input_manager *im, uint32_t buttons = SDL_GetMouseState(&mouse_x, &mouse_y); struct sc_mouse_scroll_event evt = { - .position = { - .screen_size = im->screen->frame_size, - .point = sc_screen_convert_window_to_frame_coords(im->screen, - mouse_x, mouse_y), - }, + .position = sc_input_manager_get_position(im, mouse_x, mouse_y), #if SDL_VERSION_ATLEAST(2, 0, 18) .hscroll = CLAMP(event->preciseX, -1.0f, 1.0f), .vscroll = CLAMP(event->preciseY, -1.0f, 1.0f), diff --git a/app/src/screen.c b/app/src/screen.c index 9ee61383c..11b4f58a1 100644 --- a/app/src/screen.c +++ b/app/src/screen.c @@ -962,6 +962,8 @@ sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event) { struct sc_point sc_screen_convert_drawable_to_frame_coords(struct sc_screen *screen, int32_t x, int32_t y) { + assert(screen->video); + enum sc_orientation orientation = screen->orientation; int32_t w = screen->content_size.width; ```

Please test this new binary:

scrcpy-pr4868.3.zip _{SHA-256: 1c1dea45e87ea101c29a314dd4e4994b12ce33aecbb95317bbfcd4ff642d365}

AnshulJ999 commented 2 weeks ago

Thank you! It's working now. And yes I'm on Windows 10.

Just wanted to clarify a few things:

So the scrcpy -KM --no-video --no-audio command basically is like OTG mode, but with the UHID keyboard/mouse.

The --no-window option removes the scrcpy window, so it is only used for audio playback. Nothing else seems to work.

But as you mentioned, the control events are not possible without a window? So --no-window can't be combined with -KM it seems.

Anyway thank you, I will check if I can assign a hotkey with AHK or EventGhost and make this suit my workflow a bit more.

rom1v commented 2 weeks ago

So the scrcpy -KM --no-video --no-audio command basically is like OTG mode, but with the UHID keyboard/mouse.

The main purpose of OTG mode is to work without adb (USB debugging), not to only control the device.

With this PR, if you want to only control the device and have USB debugging enabled, you can.

OTG uses AOA, so the most similar command would be:

scrcpy --keyboard=aoa --mouse=aoa --no-video --no-audio

But it is better to use UHID:

scrcpy --keyboard=uhid --mouse=uhid --no-video --no-audio
scrcpy -KM --no-video --no-audio

The --no-window option removes the scrcpy window, so it is only used for audio playback.

In the current stable version, several features do not open a window at all: audio-playback only, recording, v4L2. With this PR, these features now open a window, but there's a new option --no-window to restore the previous behavior. No window implies no control.

ristomatti commented 2 weeks ago

The options --window-height doesn't seem to work with this option, at least not with -KM --no-video --no-audio. My use case involves moving scrcpy's window off screen with just 1px shown. This creates an indicator where my phone is physically located next to the my monitor. 🙂

It works if I start with --keyboard aoa --mouse aoa --no-video --no-audio --otg --window-height=600. For me on Linux, this gives almost the same end result, except --legacy-paste only works when running with -KM --no-video --no-audio. Copy paste doesn't work without --legacy-paste on my setup. My phone is a Samsung Galaxy S23 / Android 14 (non-rooted).

ristomatti commented 2 weeks ago

there's a new option --no-window to restore the previous behavior. No window implies no control.

Is there a reason why this mode has to imply --no-control? If it didn't, it would be possible to run scrcpy from a keyboard shortcut to control the Android device and then kill the process using an OS shortcut. :grimacing: (although it might be that scrcpy would not then grab the keyboard and mouse, defeating the intended use)

rom1v commented 2 weeks ago

The options --window-height doesn't seem to work with this option

Thank you for the report. Fixed:

scrcpy-pr4868.4.zip _{SHA-256: b769c9bc403d91d78fae3efa87856dbe2af5905c8d3a33392beaf5fa7103c30}

except --legacy-paste only works when running with …

Yes, legacy paste processing is performed on the server side, and there is no server in OTG mode.

(although it might be that scrcpy would not then grab the keyboard and mouse, defeating the intended use)

That's it.

ristomatti commented 2 weeks ago

@rom1v Confirm, fixed. Thank you!

rom1v commented 6 days ago

I fixed some bugs and merged into dev.

Genymobile / scrcpy

Add scrcpy window without video playback #4868