search

Genymobile / scrcpy

Display and control your Android device
Apache License 2.0
113.14k stars 10.8k forks source link

SAMSUNG A72 - no 60fps from camera capture #4675

Open mvbhz opened 9 months ago

mvbhz commented 9 months ago

Environment

Describe the bug A clear and concise description of what the bug is.

On errors, please provide the output of the console (and adb logcat if relevant).

Please paste terminal output in a code block.

Please do not post screenshots of your terminal, just post the content as text instead.

I have a problem with capturing video from the camera of the SAMSUNG A72 phone. It constantly limits fps to 30. The camera can work at 60fps but I cannot force the program to capture at 60fps from the camera. I tried adding the command --camera-fps=60 but --print-fps still shows that it is 30fps and the video is not smooth.

rom1v commented 9 months ago

What is the result of scrcpy --list-camera-sizes?

mvbhz commented 9 months ago 
--camera-id=0    (back, 4624x3468, fps=[15, 20, 30])
    - 4000x3000
    - 4032x3024
    - 3264x2448
    - 3280x2464
    - 4032x1908
    - 3024x3024
    - 3840x2160
    - 3264x1836
    - 2576x1932
    - 1920x1080
    - 1280x720
    - 3264x1468
    - 2400x1080
    - 1440x1080
    - 1280x960
    - 1088x1088
    - 960x720
    - 720x480
    - 640x480
    - 352x288
    - 320x240
    - 176x144
  High speed capture (--camera-high-speed):
    - 1280x720 (fps=[120, 240])
    - 1920x1080 (fps=[120, 240])
--camera-id=1    (front, 3264x2448, fps=[15, 20, 30])
    - 3264x2448
    - 3264x1836
    - 3264x1468
    - 2448x2448
    - 3184x1792
    - 3184x1432
    - 2384x2384
    - 2640x1980
    - 2640x1488
    - 2640x1188
    - 1984x1984
    - 3264x1836
    - 2576x1932
    - 1920x1080
    - 1280x720
    - 3264x1468
    - 2400x1080
    - 1440x1080
    - 1280x960
    - 1088x1088
    - 960x720
    - 720x480
    - 640x480
    - 352x288
    - 320x240
    - 176x144
  High speed capture (--camera-high-speed):
    - 1280x720 (fps=[120])
    - 1920x1080 (fps=[120])
--camera-id=2    (back, 4000x3000, fps=[15, 20, 30])
    - 4000x3000
    - 4000x2256
    - 2992x2992
    - 4000x1800
    - 2560x1920
    - 2560x1440
    - 2560x1152
    - 1920x1920
    - 3840x2160
    - 3264x1836
    - 2576x1932
    - 1920x1080
    - 1280x720
    - 3264x1468
    - 2400x1080
    - 1440x1080
    - 1280x960
    - 1088x1088
    - 960x720
    - 720x480
    - 640x480
    - 352x288
    - 320x240
    - 176x144
--camera-id=3    (front, 2640x1980, fps=[15, 20, 30])
    - 2640x1980
    - 2640x1488
    - 2640x1188
    - 2576x1932
    - 1920x1080
    - 1280x720
    - 2400x1080
    - 1440x1080
    - 1280x960
    - 1088x1088
    - 960x720
    - 720x480
    - 640x480
    - 352x288
    - 320x240
    - 176x144
  High speed capture (--camera-high-speed):
    - 1280x720 (fps=[120])
    - 1920x1080 (fps=[120])

The phone has the SAMSUNG Camera application that records at 60 fps. I don't know why the program doesn't display the 60fps option. This is very strange. Can this be fixed?

rom1v commented 9 months ago 
--camera-id=0    (back, 4624x3468, fps=[15, 20, 30])

It seems your camera cannot capture at 60fps (it declares at most 30fps without high speed mode).

You can capture at 120ps in high speed mode for specific sizes though:

scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=120 --print-fps

Depending on your device, it might work to pass a non-declared value:

scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=60 --print-fps

Otherwise, you can then limit the encoding frame rate using --max-fps=60.

mvbhz commented 9 months ago

This is impossible, because, as I have already written, in the SAMSUNG Camera application I can normally record video at 60fps. After uploading this video to my computer I have real 60fps. So it's impossible for the phone's camera not to have 60fps.

rom1v commented 9 months ago

So it's impossible for the phone's camera not to have 60fps

What is the result of each of these commands:

scrcpy --video-source=camera --camera-size=1920x1080 --camera-fps=60 --print-fps
scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=60 --print-fps
scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=120 --max-fps=60 --print-fps

?

mvbhz commented 9 months ago 
C:\scrcpy-win64-v2.3.1>scrcpy --video-source=camera --camera-size=1920x1080 --camera-fps=60 --print-fps
scrcpy 2.3.1 <https://github.com/Genymobile/scrcpy>
INFO: Camera video source: control disabled
INFO: Camera video source: microphone audio source selected
INFO: ADB device found:
INFO:     -->   (usb)           R58R3249Y8Z            device  SM_A725F
C:\scrcpy-win64-v2.3.1\scrcpy-server: 1 file pushed, 0 skipped. 54.8 MB/s (66007 bytes in 0.001s)
[server] INFO: Device: [samsung] samsung SM-A725F (Android 13)
INFO: [server] INFO: Using camera '0'
Renderer: direct3d
INFO: Texture: 1920x1080
INFO: FPS counter started
INFO: 28 fps (+1 frames skipped)
INFO: 30 fps
INFO: 30 fps
C:\scrcpy-win64-v2.3.1>scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=60 --print-fps
scrcpy 2.3.1 <https://github.com/Genymobile/scrcpy>
INFO: Camera video source: control disabled
INFO: Camera video source: microphone audio source selected
INFO: ADB device found:
INFO:     -->   (usb)           R58R3249Y8Z            device  SM_A725F
C:\scrcpy-win64-v2.3.1\scrcpy-server: 1 file pushed, 0 skipped. 90.1 MB/s (66007 bytes in 0.001s)
[server] INFO: Device: [samsung] samsung SM-A725F (Android 13)
INFO: Renderer: direct3d
[server] INFO: Using camera '0'
INFO: Texture: 1920x1080
[server] ERROR: Encoding error: java.lang.IllegalArgumentException: Fps range [60, 60] in the request is not a supported high speed fps range [[120, 120], [240, 240], [30, 120], [30, 240]]
WA[server] ERROR: Exception on thread Thread[video,5,main]
Rjava.lang.IllegalArgumentException: Fps range [60, 60] in the request is not a supported high speed fps range [[120, 120], [240, 240], [30, 120], [30, 240]]
N       at android.hardware.camera2.utils.SurfaceUtils.checkConstrainedHighSpeedSurfaces(SurfaceUtils.java:253)
:       at android.hardware.camera2.impl.CameraConstrainedHighSpeedCaptureSessionImpl.createHighSpeedRequestList(CameraConstrainedHighSpeedCaptureSessionImpl.java:99)
        at com.genymobile.scrcpy.FakeContext$$ExternalSyntheticApiModelOutline0.m(Unknown Source:0)
D       at com.genymobile.scrcpy.CameraCapture.setRepeatingRequest(CameraCapture.java:340)
e       at com.genymobile.scrcpy.CameraCapture.start(CameraCapture.java:200)
v       at com.genymobile.scrcpy.SurfaceEncoder.streamScreen(SurfaceEncoder.java:72)
i       at com.genymobile.scrcpy.SurfaceEncoder.lambda$start$0$com-genymobile-scrcpy-SurfaceEncoder(SurfaceEncoder.java:253)
c       at com.genymobile.scrcpy.SurfaceEncoder$$ExternalSyntheticLambda0.run(Unknown Source:4)
e       at java.lang.Thread.run(Thread.java:1012)
 disconnected
C:\scrcpy-win64-v2.3.1>scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=120 --max-fps=60 --print-fps
scrcpy 2.3.1 <https://github.com/Genymobile/scrcpy>
INFO: Camera video source: control disabled
INFO: Camera video source: microphone audio source selected
INFO: ADB device found:
INFO:     -->   (usb)           R58R3249Y8Z            device  SM_A725F
C:\scrcpy-win64-v2.3.1\scrcpy-server: 1 file pushed, 0 skipped. 82.0 MB/s (66007 bytes in 0.001s)
[server] INFO: Device: [samsung] samsung SM-A725F (Android 13)
INFO: Renderer: direct3d
[server] INFO: Using camera '0'
INFO: Texture: 1920x1080
INFO: FPS counter started
INFO: 10 fps (+1 frames skipped)
INFO: 12 fps
INFO: 12 fps
INFO: 13 fps
mvbhz commented 9 months ago

Screenshot_20240217_165141_Camera

rom1v commented 9 months ago

Just to confirm, is the video recorded by your camera app actually at 60fps? (I think it is)

What about:

scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=120 --print-fps
scrcpy --video-source=camera --camera-size=1280x720 --camera-high-speed --camera-fps=120 --print-fps

?

mvbhz commented 9 months ago 
C:\scrcpy-win64-v2.3.1>scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=120 --print-fps
scrcpy 2.3.1 <https://github.com/Genymobile/scrcpy>
INFO: Camera video source: control disabled
INFO: Camera video source: microphone audio source selected
INFO: ADB device found:
INFO:     -->   (usb)           R58R3249Y8Z            device  SM_A725F
C:\scrcpy-win64-v2.3.1\scrcpy-server: 1 file pushed, 0 skipped. 56.7 MB/s (66007 bytes in 0.001s)
[server] INFO: Device: [samsung] samsung SM-A725F (Android 13)
[server] INFO: Using camera '0'
INFO: Renderer: direct3d
INFO: Texture: 1920x1080
INFO: FPS counter started
INFO: 11 fps (+2 frames skipped)
INFO: 13 fps
INFO: 14 fps
C:\scrcpy-win64-v2.3.1>scrcpy --video-source=camera --camera-size=1280x720 --camera-high-speed --camera-fps=120 --print-fps
scrcpy 2.3.1 <https://github.com/Genymobile/scrcpy>
INFO: Camera video source: control disabled
INFO: Camera video source: microphone audio source selected
INFO: ADB device found:
INFO:     -->   (usb)           R58R3249Y8Z            device  SM_A725F
C:\scrcpy-win64-v2.3.1\scrcpy-server: 1 file pushed, 0 skipped. 55.7 MB/s (66007 bytes in 0.001s)
[server] INFO: Device: [samsung] samsung SM-A725F (Android 13)
INFO:[server] INFO: Using camera '0'
 Renderer: direct3d
INFO: Texture: 1280x720
INFO: FPS counter started
INFO: 11 fps (+2 frames skipped)
INFO: 13 fps
INFO: 13 fps

In both cases the FPS is very low.

rom1v commented 9 months ago

Have you tried another encoder?

mvbhz commented 8 months ago

In H264 and H265 it is still only 30fps. AV1 is not working, I have a message:

ERROR: [FFmpeg] Your platform doesn't support hardware accelerated AV1 decoding. ERROR: [FFmpeg] Failed to get pixel format. ERROR: Decoder 'video': could not send video packet: -40 ERROR: Demuxer error WARN: Killing the server...

rom1v commented 8 months ago

AV1 is not working

Yes, the FFmpeg build for scrcpy for Windows does not include a AV1 decoder: https://github.com/Genymobile/scrcpy/issues/4744#issuecomment-1985629311

But in practice AFAIK no device have a correct AV1 hardware encoder anyway (in term of performance, stability and latency).

In H264 and H265 it is still only 30fps.

OK, I'm sorry, the API reports that the camera does not support 60 fps, I don't know how your samsung camera app captures it. Do you manage to record at 60fps with another camera app (like OpenCamera)?

mvbhz commented 8 months ago

I also don't know how it happens that the original camera application records at 60fps. I set 60fps in Open Camera but it records at 30fps.

parkerlreed commented 7 months ago

@rom1v Moving over here:

Tried Manual Camera Pro. 1080 and settings 60 FPS in the settings results in "Serious Camera Error", so yeah seems like 3rd party applications don't have access.

As for scrcpy

120 1080 black screen streaming

[parker@rogally ~]$ scrcpy --video-source=camera --camera-size=1920x1080 --camera-high-speed --camera-fps=120 --print-fps
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: Camera video source: control disabled
INFO: Camera video source: microphone audio source selected
INFO: ADB device found:
INFO:     -->   (usb)  R5CNB05JENH                     device  SM_G986U1
/usr/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 290.5 MB/s (163648 bytes in 0.001s)
[server] INFO: Device: [samsung] samsung SM-G986U1 (Android 13)
INFO: Renderer: opengl
INFO: OpenGL version: 4.6 (Compatibility Profile) Mesa 24.0.5-arch1.1
INFO: Trilinear filtering enabled
[server] INFO: Using camera '0'
INFO: Texture: 1920x1080
[server] WARN: Camera capture failed: frame 0
[server] WARN: Camera capture failed: frame 1
INFO: FPS counter started
INFO: 119 fps (+6 frames skipped)
INFO: 120 fps
INFO: 119 fps
INFO: 120 fps
INFO: 119 fps
INFO: 119 fps
INFO: 119 fps
INFO: 120 fps
INFO: 120 fps
INFO: 120 fps
WARN: Killing the server...

Regular 60

[parker@rogally ~]$ scrcpy --video-source=camera --camera-size=1920x1080 --camera-fps=60 --print-fps
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: Camera video source: control disabled
INFO: Camera video source: microphone audio source selected
INFO: ADB device found:
INFO:     -->   (usb)  R5CNB05JENH                     device  SM_G986U1
/usr/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 293.7 MB/s (163648 bytes in 0.001s)
[server] INFO: Device: [samsung] samsung SM-G986U1 (Android 13)
INFO: Renderer: opengl
INFO: OpenGL version: 4.6 (Compatibility Profile) Mesa 24.0.5-arch1.1
INFO: Trilinear filtering enabled
[server] INFO: Using camera '0'
INFO: Texture: 1920x1080
[server] WARN: Camera capture failed: frame 0
[server] ERROR: Audio capture error
java.io.IOException: Could not read audio: 0
        at com.genymobile.scrcpy.AudioEncoder.inputThread(AudioEncoder.java:95)
        at com.genymobile.scrcpy.AudioEncoder.lambda$encode$1$com-genymobile-scrcpy-AudioEncoder(AudioEncoder.java:195)
        at com.genymobile.scrcpy.AudioEncoder$$ExternalSyntheticLambda1.run(Unknown Source:4)
        at java.lang.Thread.run(Thread.java:1012)
[server] WARN: Camera capture failed: frame 1
[server] WARN: Camera capture failed: frame 2
[server] WARN: Camera capture failed: frame 3
[server] WARN: Camera capture failed: frame 4
[server] WARN: Camera capture failed: frame 5
[server] WARN: Camera capture failed: frame 6
[server] WARN: Camera capture failed: frame 7
[server] WARN: Camera capture failed: frame 8
[server] WARN: Camera capture failed: frame 9
[server] WARN: Camera capture failed: frame 10
[server] WARN: Camera capture failed: frame 11
[server] WARN: Camera capture failed: frame 12
[server] WARN: Camera capture failed: frame 13
[server] WARN: Camera capture failed: frame 14
[server] WARN: Camera capture failed: frame 15
[server] WARN: Camera capture failed: frame 16
[server] WARN: Camera capture failed: frame 17
[server] WARN: Camera capture failed: frame 18
[server] WARN: Camera capture failed: frame 19
[server] WARN: Camera capture failed: frame 20
[server] WARN: Camera capture failed: frame 21
parkerlreed commented 7 months ago

Seems this is purposeful on Samsung's part...

https://forum.developer.samsung.com/t/high-speed-video-recording-issue-on-samsung-s23-series/25983

https://eu.community.samsung.com/t5/galaxy-s22-series/petition-samsung-unblock-60fps-to-third-party-developers-can-we/td-p/5744386

dxzdxz1 commented 6 months ago

Seems this is purposeful on Samsung's part...

https://forum.developer.samsung.com/t/high-speed-video-recording-issue-on-samsung-s23-series/25983

https://eu.community.samsung.com/t5/galaxy-s22-series/petition-samsung-unblock-60fps-to-third-party-developers-can-we/td-p/5744386

Yes, it is exactly this. There's a list of allowed packages that can use full camera capabilities. I don't know the full list, but modders use com.samsung.android.scan3d and com.samsung.android.ruler as the package names for their camera mods.

With Camera2Test we can see the difference. Using a modified Camera2Test app with com.samsung.android.scan3d package name on my S23Ultra, I can see that Samsung exposes more Camera IDs, such as ID 5, 6, 7, 20, 21, 23, 52, 54, 56 and 58, none of them are accessible from non-samsung apps, they can only see ID 0, 1, 2 and 3.

I don't know anything about programming or anything like that, so I may be wrong, but since scrcpy uses ADB to connect to devices, I don't think it's possible to spoof the access to the camera as if it was from a Samsung app, but most probably it can't. Maybe it's possible from a rooted device if the person knows where to edit the list of allowed apps.

rom1v commented 6 months ago

There's a list of allowed packages that can use full camera capabilities. I don't know the full list, but modders use com.samsung.android.scan3d and com.samsung.android.ruler as the package names for their camera mods.

You can try to change the value here:

https://github.com/Genymobile/scrcpy/blob/206809a99affad9a7aa58fcf7593cea71f48954d/server/src/main/java/com/genymobile/scrcpy/FakeContext.java#L12

dxzdxz1 commented 6 months ago

There's a list of allowed packages that can use full camera capabilities. I don't know the full list, but modders use com.samsung.android.scan3d and com.samsung.android.ruler as the package names for their camera mods.

You can try to change the value here:

https://github.com/Genymobile/scrcpy/blob/206809a99affad9a7aa58fcf7593cea71f48954d/server/src/main/java/com/genymobile/scrcpy/FakeContext.java#L12

From my understanding, for this to work I have to install the Android Shell apk with a modified package name, right? A few hours ago, I clone and build scrcpy modifying this line you mentioned and replacing with com.samsung.android.scan3d. Then I tried to rename the Android Shell package name to com.samsung.android.scan3d and Install, but Android doesn't let me install, it says that my package conflicts with an existing package (even though the package name is different), I don't know any way to bypass this.

So I googled for an alternative shell for Android, and I found this one https://smartpack.github.io/ashell

Now my question is, is it possible to run scrcpy through aShell (even if I need to start the scrcpy server directly from my phone) and communicate with the client on Windows? I contacted aShell developer on Telegram and he said that he would help if needed

I don't know if I'm going too far, but this could be beneficial to all Samsung phone users that uses scrcpy

Also, and most probably, I'm all wrong because I don't understand much about these things, english is not my main language, and I'm basically googling everything so far lol

rom1v commented 6 months ago

You don't need to install it, "just" rebuild scrcpy with that change and run it.

https://github.com/Genymobile/scrcpy/blob/master/doc/build.md

dxzdxz1 commented 6 months ago

You don't need to install it, "just" rebuild scrcpy with that change and run it.

https://github.com/Genymobile/scrcpy/blob/master/doc/build.md

I did this, I got some warnings during the process (the main exe and server were created anyway). Unfortunately, it didn't work. I got the same results

Building with msys2: ![Screenshot 2024-05-13 174232](https://github.com/Genymobile/scrcpy/assets/5876517/a2ec6033-01bf-4d6e-87fe-b57b9caa742f)
Running custom build: ![](https://github.com/Genymobile/scrcpy/assets/5876517/4441daa1-2427-40c7-ac19-9e67678c66a6)
parkerlreed commented 1 week ago

Yeah sadly I confirmed the build worked but doesn't present any new modes

[parker@parker-framework server]$ grep -r ruler scrcpy-server.out/
scrcpy-server.out/smali/com/genymobile/scrcpy/audio/AudioDirectCapture.smali:    const-string v2, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/audio/AudioDirectCapture.smali:    const-string v1, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/Workarounds.smali:    const-string v4, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/FakeContext.smali:.field public static final PACKAGE_NAME:Ljava/lang/String; = "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/FakeContext.smali:    const-string v1, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/FakeContext.smali:    const-string v0, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/FakeContext.smali:    const-string v0, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/wrappers/ClipboardManager.smali:    const-string v2, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/wrappers/ClipboardManager.smali:    const-string v2, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/wrappers/ClipboardManager.smali:    const-string v2, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/wrappers/ContentProvider.smali:    const-string v2, "com.samsung.android.ruler"
scrcpy-server.out/smali/com/genymobile/scrcpy/wrappers/ActivityManager.smali:    const-string v5, "com.samsung.android.ruler"
[parker@parker-framework server]$ cd ../..
[parker@parker-framework scrcpy]$ ./run x --list-cameras
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  R3CR700V30T                     device  SM_F711U1
x/server/scrcpy-server: 1 file pushed, 0 skipped. 166.3 MB/s (71200 bytes in 0.000s)
[server] INFO: Device: [samsung] samsung SM-F711U1 (Android 14)
[server] INFO: List of cameras:
    --camera-id=0    (back, 4000x3000, fps=[10, 15, 24, 30])
    --camera-id=1    (front, 3648x2736, fps=[10, 15, 24, 30])
    --camera-id=2    (back, 4000x3000, fps=[10, 15, 24, 30])
    --camera-id=3    (front, 3216x2208, fps=[10, 15, 24, 30])
parkerlreed commented 1 week ago

Even worse with the changed package name it never gets a camera stream even on the supported modes.

EDIT: So the faked package name CAN get a video stream on Android 13. Breaks on Android 14.

Even with the faked package name on Android 13, no extra modes are presented and manually trying 60 FPS results in the Camera Capture Failed.