Developer Advice: Simulate multitouch with key or mouse

[andresUC-GCS]

Hi, I'm a game developer and my QA teammate uses scrcpy to test our games in multiple remote phones with different specs. This is working nice so far, but we just have one issue: In this game (which is in a develop version) you can open the "cheat console" with a 3-finger touch, this is an important feature for the QA team because it helps them to quickly test their cases.

The PC doesn't have a touch screen (I read that multitouch is possible with it), so what we'd like is to assign some key or key-mouse combination to achieve this. I know this is a very specific feature, so I was thinking I can implement it myself locally (without making a PR) but first I want to know if that implementation is possible, and if it's possible I would like an advice on which files I would need to modify.

Thanks in advance.

[rom1v]

To inject an event, call sc_controller_push_msh(). Here is an example to inject a touch event:

https://github.com/Genymobile/scrcpy/blob/c083a7cc9015b52e12b0fb78e5a488ab1f5b68f6/app/src/mouse_inject.c#L131-L144

You should probably call it from input_manager.c, keyboard_inject.c or mouse_inject.c.

[andresUC-GCS]

Thanks! I'll give it a try. Do I have to send 3 touch events or is there a property in the event to configure the number of fingers?

[rom1v]

3 successive sc_touch_events having different pointer_ids with action SC_TOUCH_ACTION_DOWN, then 3 successive sc_touch_events with action SC_TOUCH_ACTION_UP (having the same 3 pointer_ids as for the down action).

[andresUC-GCS]

Hi again, I finished making the changes I needed but then had trouble trying to build. I'm trying to do the cross-compile with a Debian VM and according to the build instructions I don't need Android SDK if I use the prebuilt server, but the build was still failing and complaining about it. Then I installed Android studio and set the root sdk but I'm getting the same error.

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':server:compileDebugJavaWithJavac'.
> SDK location not found. Define a valid SDK location with an ANDROID_HOME environment variable or by setting the sdk.dir path in your project's local properties file at '/home/friday/RiderProjects/scrcpy/local.properties'.

I think I'm doing something wrong, or do I need to build the server too because of the changes I made?

EDIT: I already downloaded the prebuilt server and ran these commands replacing the path with the actual path of the dowloaded zip file:

meson setup x --buildtype=release --strip -Db_lto=true \
    -Dprebuilt_server=/path/to/scrcpy-server
ninja -Cx  # DO NOT RUN AS ROOT

[Termiiii]

I think I'm doing something wrong, or do I need to build the server too because of the changes I made?

Its the same Error I had for multiple hours. My setup: Windows PC with Debian Subsystem (basically installed like this: https://www.youtube.com/watch?v=JzAAxL3HrXU) I used the build guide here: https://github.com/Genymobile/scrcpy/blob/master/doc/build.md

I had problems installing Java11, I ended up downloading it from here: https://jdk.java.net/archive/ and following this guide for installing Java: https://www.youtube.com/watch?v=72mz0-98vg8

The next error happened, with the missing android-sdk. My error happened with this command:

./release.sh

first, I realized it should be: sudo ./release.sh instead. (ignore this) And then I realized it wanted the android-sdk, which the guide did not specify before. The guide mentioned android sdk platform tools, which is sadly not the same.

I followed the following guide to get android-sdk: https://www.youtube.com/watch?v=OGJOPnV2fDc I got the android-sdk by installing android studio, going into the settings and copying the path from it, and putting it inside the local.properties file like this:

sdk.dir=/home/termii/Android/Sdk

Note that if your android sdk path starts with root/*, you need to reinstall android studio without sudo.

[rom1v]

@Termiiii What are you trying to do? Is it related to this issue?

Windows PC with Debian Subsystem

( have never tested)

first, I realized it should be sudo ./release.sh

No, it should not. Do not execute that script as root.

If you just want to build the latest release, you can use the prebuilt server so you don't need the Android SDK at all.

Otherwise, yes, you need the Android SDK.

[Termiiii]

I tried to build client & server, because I want to modify the client later on. I am on Windows, so I assumed I need to follow the "Cross-compile from Linux"-section (assuming it is for Windows + Debian Subsystem). I believe it is related, because my error message was the same as from @andresUC-GCS

Cross-compile from Linux

This is the preferred method (and the way the release is built).

From Debian, install mingw: sudo apt install mingw-w64 mingw-w64-tools

You also need the JDK to build the server: sudo apt install openjdk-11-jdk

Then generate the releases: ./release.sh

It will generate win32 and win64 releases into dist/.

If I don't execute the script as root, I get all kinds of errors:

termii@DESKTOP-PKFAPQQ:~/my$ cd scrcpy
termii@DESKTOP-PKFAPQQ:~/my/scrcpy$ ./release.sh
./gradlew clean
> Task :server:clean FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':server:clean'.
> java.io.IOException: Unable to delete directory '/home/termii/my/scrcpy/server/build'
    Failed to delete some children. This might happen because a process has files open or has its working directory set in the target directory.
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/release/com/genymobile/scrcpy/BuildConfig.java
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/release/com/genymobile/scrcpy
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/release/com/genymobile
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/release/com
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/release
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/debug/com/genymobile/scrcpy/BuildConfig.java
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/debug/com/genymobile/scrcpy
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/debug/com/genymobile
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/debug/com
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig/debug
    - /home/termii/my/scrcpy/server/build/generated/source/buildConfig
    - /home/termii/my/scrcpy/server/build/generated/source
    - /home/termii/my/scrcpy/server/build/generated/res/resValues/androidTest/debug
    - /home/termii/my/scrcpy/server/build/generated/res/resValues/androidTest
    - /home/termii/my/scrcpy/server/build/generated/res/resValues/release
    - /home/termii/my/scrcpy/server/build/generated/res/resValues/debug
    - and more ...

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
2 actionable tasks: 1 executed, 1 up-to-date
make: *** [release.mk:50: clean] Error 1
termii@DESKTOP-PKFAPQQ:~/my/scrcpy$

I don't know if my root approach is working either, I am currently wondering what the current error is:

termii@DESKTOP-PKFAPQQ:~/my/scrcpy$ sudo ./release.sh
[sudo] password for termii:
./gradlew clean

BUILD SUCCESSFUL in 1s
2 actionable tasks: 1 executed, 1 up-to-date
rm -rf "dist" "build-test" "build-server" \
        "build-win32" "build-win64"
[ -d "build-test" ] || ( mkdir "build-test" && \
        meson setup "build-test" -Db_sanitize=address )
The Meson build system
Version: 1.0.1
Source dir: /home/termii/my/scrcpy
Build dir: /home/termii/my/scrcpy/build-test
Build type: native build
Project name: scrcpy
Project version: 2.1.1
C compiler for the host machine: cc (gcc 12.2.0 "cc (Debian 12.2.0-14) 12.2.0")
C linker for the host machine: cc ld.bfd 2.40
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (1.8.1)
Run-time dependency libavformat found: YES 59.27.100
Run-time dependency libavcodec found: YES 59.37.100
Run-time dependency libavutil found: YES 57.28.100
Run-time dependency libswresample found: YES 4.7.100
Run-time dependency sdl2 found: YES 2.26.5
Run-time dependency libavdevice found: YES 59.7.100
Run-time dependency libusb-1.0 found: YES 1.0.26
Checking for function "strdup" : YES
Checking for function "asprintf" : YES
Checking for function "vasprintf" : YES
Checking for function "nrand48" : YES
Checking for function "jrand48" : YES
Checking for function "reallocarray" : YES
Header "sys/socket.h" has symbol "SOCK_CLOEXEC" : YES
Configuring config.h using configuration
Program ./scripts/build-wrapper.sh found: YES (/home/termii/my/scrcpy/server/./scripts/build-wrapper.sh)
Build targets in project: 13

scrcpy 2.1.1

  User defined options
    b_sanitize: address

Found ninja-1.11.1 at /usr/bin/ninja
ninja -C "build-test"
ninja: Entering directory `build-test'
[88/114] Generating server/scrcpy-server with a custom command
(not invoking gradle, since we are root)
[114/114] Linking target app/scrcpy
./gradlew -p server check

> Task :server:lintReportDebug
Wrote HTML report to file:///home/termii/my/scrcpy/server/build/reports/lint-results-debug.html

BUILD SUCCESSFUL in 10s
45 actionable tasks: 45 executed
[ -d "build-server" ] || ( mkdir "build-server" && \
        meson setup "build-server" --buildtype release -Dcompile_app=false )
The Meson build system
Version: 1.0.1
Source dir: /home/termii/my/scrcpy
Build dir: /home/termii/my/scrcpy/build-server
Build type: native build
Project name: scrcpy
Project version: 2.1.1
C compiler for the host machine: cc (gcc 12.2.0 "cc (Debian 12.2.0-14) 12.2.0")
C linker for the host machine: cc ld.bfd 2.40
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program ./scripts/build-wrapper.sh found: YES (/home/termii/my/scrcpy/server/./scripts/build-wrapper.sh)
Build targets in project: 2

scrcpy 2.1.1

  User defined options
    buildtype  : release
    compile_app: false

Found ninja-1.11.1 at /usr/bin/ninja
ninja -C "build-server"
ninja: Entering directory `build-server'
[0/1] Generating server/scrcpy-server with a custom command
(not invoking gradle, since we are root)
platform-tools-34.0.3 found
SDL2-2.28.0 found
ffmpeg-6.0-scrcpy-4 found
libusb-1.0.26 found
[ -d "build-win32" ] || ( mkdir "build-win32" && \
        meson setup "build-win32" \
                --cross-file cross_win32.txt \
                --buildtype release --strip -Db_lto=true \
                -Dcompile_server=false \
                -Dportable=true )
The Meson build system
Version: 1.0.1
Source dir: /home/termii/my/scrcpy
Build dir: /home/termii/my/scrcpy/build-win32
Build type: cross build
Project name: scrcpy
Project version: 2.1.1
C compiler for the host machine: i686-w64-mingw32-gcc (gcc 12.0.0 "i686-w64-mingw32-gcc (GCC) 12-win32")
C linker for the host machine: i686-w64-mingw32-gcc ld.bfd 2.40
C compiler for the build machine: cc (gcc 12.2.0 "cc (Debian 12.2.0-14) 12.2.0")
C linker for the build machine: cc ld.bfd 2.40
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86
Host machine cpu: i686
Target machine cpu family: x86
Target machine cpu: i686
Windows resource compiler: GNU windres (GNU Binutils) 2.40
Library SDL2 found: YES
Library SDL2main found: YES

app/meson.build:141:4: ERROR: C shared or static library 'avcodec-60' not found

A full log can be found at /home/termii/my/scrcpy/build-win32/meson-logs/meson-log.txt
make: *** [release.mk:72: build-win32] Error 1

[rom1v]

If I don't execute the script as root, I get all kinds of errors:

Because you executed with sudo once, so it created the directories with root permissions. You should delete them (as root) and retry (without root).

app/meson.build:141:4: ERROR: C shared or static library 'avcodec-60' not found

When building from Windows, this dependency is downloaded and extracted to app/prebuilt-deps/data. What's the content of this folder (and the ffmpeg one inside it)? If you remove it, it will re-download-extract it.

[Termiiii]

thx for your help, I will do that later. This is the content, am I supposed to only remove ffmpeg or everything? Edit: both the ffmpeg and libusb folders were empty. Deleting them worked, thanks!