Scrcpy image freezes after some frames

[Slluxx]

• [x] I have read the FAQ.
• [x] I have searched in existing issues.

Environment

• OS: Windows 7
• scrcpy version: 1.21 (1.20 same issue)
• installation method: windows release
• device model: Xiaomi Redmi Note 10 Pro
• Android version: 12 (ArrowOs [aosp] custom rom)

Describe the bug First i should say that i have to use the windows 7 fix from #2840 and i also need to use --encoder OMX.google.h264.encoder or c2.qti.avc.encoder, otherwise i get errors. Using a different framerate, resolution or switching to the opengl renderer doesnt fix it.

Scrcpy opens up and everything works, till i touch my phone or use scrcpy to swipe or click. As soon as something happens on the display, scrcpy works for a few more frames and then the image (not the software) freezes. I can still input swipes in the software and they get executed on my phone but video is not transmitted anymore. It just stays at whatever frame froze.

scrcpy --record file.mp4 records untill the point where everything freezes, the ends and no further recording is saved. adb shell screenrecord sdcard/test.mp4 records for how long i need it but its all black The phone/android has an inbuild recording function which works perfectly fine

C:\Users\calvin\Desktop\scrcpy_new>scrcpy -m 1024 --encoder OMX.google.h264.encoder
scrcpy 1.21 <https://github.com/Genymobile/scrcpy>
C:\Users\calvin\Desktop\scrcpy_new\scrcpy-server: 1 file pushed, 0 skipped. 25.2 MB/s (40067 bytes in 0.002s)
[server] INFO: Device: Xiaomi M2101K6G (Android 12)
INFO: Renderer: direct3d
INFO: Initial texture: 464x1024

[rom1v]

Looks like a device/rom/encoder specific-issue :/

[Slluxx]

Ive updated my post. It records fine untill the point where it freezes. The recording stops, even though the window itself isnt closed and still shows the last frame.

Is there a way to debug the specific issue? If its really the rom, i could report it and maybe it will get fixed in the next build

[rom1v]

I guess your encoder stops producing frames: https://github.com/Genymobile/scrcpy/blob/479abc8c7723d2d05b7c5027528acd91946a328c/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L110

You could add some traces to check.

[Slluxx]

You could add some traces to check.

Now how would i do that?

[Fabxrm]

Same issue on the same phone Xiaomi Redmi Note 10 Pro (M2101K6G) after update to Android 12... let me know if you resolve thanks!

[wchensc]

It happened to me as well. I used the scrcpy java server only and do my streaming. The screen works well for a while and suddenly freezes. Control keeps working. Then after 10-30 seconds, the screen moves as normal again. But it will freeze later again. Please help and this is blocking my release. Thanks! @rom1v

scrcpy version: 1.24 Android version: 10, 11, 12 Encoder name: c2.android.avc.encoder/OMX.google.h264.encoder

[rom1v]

@wchensc If you record (scrcpy --record file.mp4) is the same problem visible on the recorded file?

Does it also happen with a lower resolution? (scrcpy -m800`)

Does it also happen on other apps on the device (other than games)?

Note that there are problems with xiaomi devices on Android 12 (like redmi note 10 pro): #3051

[wchensc]

I am not using scrcpy directly. I am only use the server part and read the video frame from the socket. So it doesn't support saving to video file. And I am using Samsung phones. @rom1v

[rom1v]

I am not using scrcpy directly. I am only use the server part and read the video frame from the socket.

If you use scrcpy, does it also freeze?

[wchensc]

I tried using scrcpy for 2-3 min and it didn't seem to freeze. Do you have any idea what could be the possible root cause? Any pointers will be helpful and I can debug myself. Thanks.

[rom1v]

Maybe you don't consume the frames quickly enough (if you perform slme long-running synchronous processing). I don't know what youbdo on the client side.

[wchensc]

Could you elaborate a bit on the don't consume the frames quickly enough? My client is simple, it just reads the data from the socket and send the frames to the browser through an RTP connection(WebRTC). What confused me is why not consuming quickly enough freezes the screen. Shouldn't it just lag for a bit and the screen moves a bit more slowly than normal? But instead it freezes for a long time. Thanks.

[rom1v]

it just reads the data from the socket and send the frames to the browser through an RTP connection(WebRTC)

Do you transmit the H.264 packets or the decoded frames (to be re-encoded by your WebRTC tools)?

What confused me is why not consuming quickly enough freezes the screen

Since the input is in real time, it cannot be paced (the back-pressure could not be propagated). If you consumes slower that what the encoder produces, the intermediate buffers (encoder, socket…) will fill up, and your hardware encoder will not be able to encode more data, so everything is blocked (in that case, it probably drops frames, until buffers are available so that it generates a new I-frame)…

[wchensc]

Do you transmit the H.264 packets or the decoded frames (to be re-encoded by your WebRTC tools)?

The client splits the bytes into NAL unit by checking the boundary(00000001) and send the bytes to the browser through the WebRTC tool. The webrtc tool will decode it.

I also found that when the screen freezes, if i swipe the screen, the client also receives the frame data as normal. It is just the screen not showing new frames.

[rom1v]

You should measure the time between when you have received a frame and the time where you start reading the next one (in other words, the time taken by I/O and processing when you call your WebRTC stuff).

[wchensc]

The longest duration I saw was around 6 mil sec, which should be very minimal.

[wchensc]

But I don't feel it is a scrcpy problem. Don't want to waste more of your time. Thanks for responding.