smooth on phone, low fps on cast screen

[Maliciousrean]

• [*] I have read the FAQ.
• [*] I have searched in existing issues.

Environment

• OS: android / win10
• scrcpy version: 1.16
• installation method: manual
• device model: xiaomi note 9s
• Android version: 10

Describe the bug casting screen of epic7, phone connected to pc through usb cable. on my phone picture is smooth but on the cast screen its choppy. phone tool shows 50% cpu/gpu load, changing bitrate, encoding to opengl, resolution does nothing. i dont understand what may cause this. fps tool shows 45-48fps, my phone shows 50fps in game. but while phone image is very smooth, cast is choppy and staggering every few seconds. video recorded through scrcpy is smooth, so i guess its a codec issue? but how do i even fix it? i have no idea where to look. my gpu is 2070 super and i have LAV installed because i have VR. so everything should be in place for it to work properly.

[rom1v]

fps tool shows 45-48fps, my phone shows 50fps in game. but while phone image is very smooth, cast is choppy and staggering every few seconds.

Any "skipped frames" reported?

video recorded through scrcpy is smooth, so i guess its a codec issue?

A decoder issue on the computer side I guess.

Out of curiosity, is OBS capturing the scrcpy window?

[Maliciousrean]

i dont have OBS launched, not streaming. there are no skipped frames either. i tried no frame skip mode and it only got worse. any idea what i can do about this situation? add: disabled nvidia shadowplay just in case, did nothing

[rom1v]

resolution does nothing

Even a tiny definition like scrcpy -m400?

[Maliciousrean]

resolution does nothing

Even a tiny definition like scrcpy -m400?

yep, still choppy

[rom1v]

OK, it's unexpected.

Could you please run scrcpy -r file.mkv and also capture a video of your computer screen, and post both video files?

[Maliciousrean]

file.zip file is attached, screen cap is on youtube "link edited for privacy reasons" you can see choppy animation that isnt present in local recording.

[rom1v]

Thank you. I got it: it's probably due to the variation of decoding time between frames. This could be confirmed by adding logs to print the decoding time.

On purpose, to minimize latency, scrcpy displays a frame as soon as it is available. But if the decoder decodes one frame in 5ms and another in 25ms (which would not be surprising, not all frames have the same complexity), then the second will be displayed 20ms too late compared to the previous one. For me, it was just a theoretical problem, since in practice I never noticed the problem (when playing a video for example).

It is a design choice to ignore network lag (since over USB it's not a problem) to optimize for latency. But it might make sense to add an option to compensate for decoder "jitter" (at the cost of increased average latency). I will think about it.

[Maliciousrean]

so for now its unavoidable? got it. regardless, very cool software, thanks for all the hard work. if you'll come up with the beta solution or smth hit me up, i'll be glad to test it for you on my end.

[rom1v]

Could you apply this patch to log the decoding time of each frame, please?

diff --git a/app/src/decoder.c b/app/src/decoder.c
index 49d4ce86..b3cf91e7 100644
--- a/app/src/decoder.c
+++ b/app/src/decoder.c
@@ -5,6 +5,7 @@
 #include <SDL2/SDL_events.h>
 #include <SDL2/SDL_mutex.h>
 #include <SDL2/SDL_thread.h>
+#include <SDL2/SDL_timer.h>
 #include <unistd.h>

 #include "config.h"
@@ -63,6 +64,8 @@ bool
 decoder_push(struct decoder *decoder, const AVPacket *packet) {
 // the new decoding/encoding API has been introduced by:
 // <http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=7fc329e2dd6226dfecaa4a1d7adf353bf2773726>
+    static unsigned i;
+    uint32_t start = SDL_GetTicks();
 #ifdef SCRCPY_LAVF_HAS_NEW_ENCODING_DECODING_API
     int ret;
     if ((ret = avcodec_send_packet(decoder->codec_ctx, packet)) < 0) {
@@ -92,6 +95,9 @@ decoder_push(struct decoder *decoder, const AVPacket *packet) {
         push_frame(decoder);
     }
 #endif
+    uint32_t end = SDL_GetTicks();
+    LOGI("packet %d decoding: %" PRIu32 "ms", i++, end - start);
+
     return true;
 }

[Maliciousrean]

uh, i dont really have git installed or know how it works. is it possible to just send me the compiled files? :< if not i'll get to figuring it out after i wake up, shouldnt be too hard

[rom1v]

Replace in v1.16 release:

• scrcpy.exe SHA256: 5a60c9decba62bdfb6a8d62702cde7d906a4f6f186949a624082d250631b13d8

[Maliciousrean]

it seems much better log of about a minute in attach log.txt

INFO: packet 0 decoding: 6ms
INFO: packet 1 decoding: 4ms
INFO: packet 2 decoding: 5ms
INFO: packet 3 decoding: 4ms
INFO: packet 4 decoding: 6ms
INFO: packet 5 decoding: 5ms
INFO: packet 6 decoding: 5ms
INFO: packet 7 decoding: 7ms
INFO: packet 8 decoding: 5ms
INFO: packet 9 decoding: 6ms
INFO: packet 10 decoding: 4ms
INFO: packet 11 decoding: 3ms
INFO: packet 12 decoding: 3ms
INFO: packet 13 decoding: 3ms
INFO: packet 14 decoding: 5ms
INFO: packet 15 decoding: 2ms
INFO: packet 16 decoding: 3ms
INFO: packet 17 decoding: 3ms
INFO: packet 18 decoding: 6ms
INFO: packet 19 decoding: 2ms
INFO: packet 20 decoding: 4ms
INFO: packet 21 decoding: 2ms
INFO: packet 22 decoding: 3ms
INFO: packet 23 decoding: 2ms
INFO: packet 24 decoding: 5ms
INFO: packet 25 decoding: 1ms
INFO: packet 26 decoding: 5ms
INFO: packet 27 decoding: 7ms
INFO: packet 28 decoding: 7ms
INFO: packet 29 decoding: 6ms
INFO: packet 30 decoding: 7ms
INFO: packet 31 decoding: 6ms
INFO: packet 32 decoding: 6ms
INFO: packet 33 decoding: 5ms
INFO: packet 34 decoding: 4ms
INFO: packet 35 decoding: 7ms
INFO: packet 36 decoding: 6ms
INFO: packet 37 decoding: 5ms
INFO: packet 38 decoding: 5ms
INFO: packet 39 decoding: 6ms
INFO: packet 40 decoding: 5ms
INFO: packet 41 decoding: 3ms
INFO: packet 42 decoding: 4ms

[rom1v]

OK, so the problem probably does not come from the decoder, your results are even better than on my laptop.

So it lags somewhere else.

Are you able to test on the same computer (or even another computer) on Linux?

[Maliciousrean]

wait but it doesnt stutter anymore. fps went down to 20s on that same screen i made video of earlier, but its no longer jittering. i am totally fine with lower fps but stable framerate rather than higher fps and jitters. here is a short vid Desktop 2020.11.15 - 21.01.47.03.DVR.mp4.zip

dont have linux unfortunately.

[rom1v]

wait but it doesnt stutter anymore

That's not expected, I didn't change anything (except adding a log).

[Maliciousrean]

okay i went back to old exe and its the same behavior, its not jittering anymore. i'm not sure what changed. i did update graphics drivers out of desperation yesterday but didnt reboot since then (it was shortly before i made the ticket). so after i went to sleep i turned off my pc. maybe that fixed it? idk, i'll keep an eye on it. maybe it was buggy nvidia driver?

[rom1v]

Yes, if the same binary now works correctly, I think it was related to graphics drivers.

[Maliciousrean]

deeply apologize for sending you on a goose chase. i guess issue is resolved then, i'll let you know if i see similar behavior again

[rom1v]

No problem, that was a legitimate issue to report 😉 (and I'm glad that it's finally just a driver issue 😄).

[Maliciousrean]

No problem, that was a legitimate issue to report 😉 (and I'm glad that it's finally just a driver issue 😄).

i restarted the game and its back. was not gpu drivers. i'm trying to mess with it for now wondering if there is any connection between when cast/program is launched, here is the video of a picture with the logging enabled. even logging is stuttering Desktop 2020.11.16 - 01.43.03.07.DVR.mp4.zip

i wonder if it could be just a software issue with the phone or the game? anyway i'm tinkering with it for now trying to figure out what did i do to make it work smooth

[Maliciousrean]

okay to recap the entire thing: after i went to sleep i left phone with the game on and turned off pc. i woke up, saw your comment, changed exe and hooked up my phone to pc (i unplugged it from usb for the night). it was working fine and that screen with the dude was showing 20fps but not stuttering. then i restarted the game and stuttering was back and alt+i now shows 45+fps again at that same exact screen.

i wonder why it was rendering at 20fps back then and without the jitter. i'll note that 45+fps is normal behavior for the game as we already established that stutter is only on pc screen

add: cant replicate success in running the game smooth. i'll try to see how it runs on my work pc later on to eliminate any pc-related issues. since issue reappeared after restarting the game while still being connected over scrcpy (i didnt unplug my phone all that time) i'm starting to think its something to do with the phone/game and how scrcpy hooks up to it. or there is something interfering with it. i tried to do cast over wi-fi, same issues. tho i guess none of this explains why it was decoding with 20fps on that screen...

[rom1v]

On your video, even your console seems to be lagging (while it shouldn't happen if the video is smooth when recorded).

Could you start an animation on the desktop while scrcpy is running (like a rotating triangle or something) to see if it lags the same way?

[Maliciousrean]

i'm usually watching twitch and i didnt notice any slowdown. and honeslty i shouldnt have granted i pack ryzen3600/2070super. randomly, again, seems fine for now. i'll let you know if other stuff is affected if it happens once more

[Maliciousrean]

0 slowdown in any other app when this happens. tried youtube and twitch, all working fine (ignoer the spike on gpu chart it only happened because i opened up a video i just recorded)

Desktop 2020.11.18 - 22.13.21.10.DVR.mp4.zip

shouldnt be related to pc load, i would've noticed that long ago. actually i just noticed, decoding stutters even when window is minimized and gpu isnt drawing anything on screen.

so far i havent been able to identify why it lags sometimes and other times its fine.

[rom1v]

Maybe the lags comes from the encoder on the device (which sets the correct PTS but provide some buffers too late) then.

[Maliciousrean]

i am also leaning towards some kind of issue on the phone itself as the bug seem to happen randomly and may occur when just restarting the app. if i'll figure it out i'll let you know

[Maliciousrean]

okay i'm at a complete loss. since last post i didnt touch my phone and it was casting for 7 hours straight while i was playing the game and afk. never switched app or anythng, game was always on. suddenly frame drops vanished. at this rate i call random phone shenanigans that are...random. impossible to debug without really digging into phone. so i guess thats it for the ticket. thanks for all the help, much appreciated.

[StacktraceException]

still occurs but on xiaomi 11t here. tried even with 1m bitrate on both usb and wifi and nothing helped.

This is the best I got actually: adb shell wm size 1440x2560 density 216 && .\scrcpy.exe --print-fps --always-on-top --shortcut-mod=lctrl+lalt --stay-awake --turn-screen-off --render-driver=metal --disable-screensaver --video-bit-rate=1M --max-fps=120 --video-codec=h265 --video-encoder='OMX.MTK.VIDEO.ENCODER.HEVC'

Using these options (especially the different encoder made the fps a lot more stable although still very poor).

[itzyashh]

I was having this issue and it got resolved after connecting an hdmi cable to tv and a mouse to operate phone ,during this process it was also connected to scrcpy using tcpip .Maybe connecting to hdmi reset something which removed all the lag .