search

dec05eba / gpu-screen-recorder-issues

GPU Screen Recorder issue tracker
13 stars 0 forks source link

[BUG] Framerate mode "constant" results in stuttering video export on Ubuntu #15

Closed Sturmlocke86 closed 4 months ago

Sturmlocke86 commented 4 months ago

Describe the bug Not sure if this is a bug or not, seeking help here. Video framerate is stuttering and unstable in certain games when using "constant" framerate for recording. I require constant framerate for video editing post recording. Some games run just fine, others are stuttering at 1 fps. Tested with both Steam + Proton as Flatpak and Snap on Ubuntu 23.10 and 22.04.

Example stuttering video: Alien Isolation, Everquest 2

Example fluid 60 fps video: Turok 2 PC remaster, Aliens vs Predator 1, Crazy Taxi

To Reproduce

See attached screenshot: I tried lowering the video quality settings to medium, changing the codec from H265 to H264, and dropping the resolution of the game to lower levels. I usually record at 4k60 resolution, but lower resolutions have the same issue. Lowering the graphics in-game to the lowest setting doesn't help the issue either.

Expected behavior I expect the video to run at buttery smooth 60 fps.

Screenshots See screenshots and video examples attached to this report.

Desktop (please complete the following information):

Additional context

flatpak run com.dec05eba.gpu_screen_recorder Gtk-Message: 21:42:45.851: Failed to load module "canberra-gtk-module" Gtk-Message: 21:42:45.852: Failed to load module "canberra-gtk-module" window name: Alien: Isolation, window id: 8388707 Warning: opus and flac audio codecs are temporary disabled, using aac audio codec instead [hevc_nvenc @ 0x603820a9c280] ignoring invalid SAR: 0/0 Info: using hevc encoder because a codec was not specified [hevc_nvenc @ 0x603820a9c280] ignoring invalid SAR: 0/0 update fps: 239 update fps: 77 update fps: 27 update fps: 17 update fps: 13 update fps: 10 update fps: 10 update fps: 9 update fps: 7 update fps: 6 update fps: 5 update fps: 4 update fps: 4 update fps: 5 update fps: 4 update fps: 4 update fps: 3 update fps: 3 update fps: 3 update fps: 3 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 2 update fps: 1 update fps: 2 update fps: 2 update fps: 2 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 update fps: 1 Notification: title: GPU Screen Recorder, body: The recording was saved to /home/sturmlocke/Videos/GPU Screen Rec/Video_2024-04-26_21-44-12.mp4

I have been recording on openSUSE Leap 15.5 with another PC that is identical to this one hardware-wise, and have not experienced these issues on openSUSE (recording with the exact same settings with constant framerate mode produces 60 fps smooth videos on Suse). On Ubuntu, recording anything outside of Steam and Proton runs just fine. I have tried recording standalone Flatpak and Snap games, such as Xonotic, and the video is super smooth at 60 fps. Worth mentioning: I've noticed that the Nvidia video engine is often at 90-100% load for the videos recorded where it's stuttering badly. In other words, 90-100% load during recording. No fps-drops in gameplay, only the video stutters post recording.

Tested with MPV player, VLC and Gnome Video player. Importing the video into Shotcut or Kdenlive video editior shows the same stuttering picture.

Feel free to ask for more infos, will gladly provide more feedback.

https://github.com/dec05eba/gpu-screen-recorder-issues/assets/86625411/020ad589-432e-469e-96cf-5d14d6747d69

Bildschirmfoto vom 2024-04-26 22-07-33

Sturmlocke86 commented 3 months ago

Thanks! Hope it doesn't cause too much trouble. I wish I had another Nvidia machine to test with a completely different gpu series, but the only other thing that I have around here right now would be from AMD. Btw, I went ahead and tried to work with VFR videos instead of doing it with CFR material because with VFR the slowdown does not happen in EQ2 (so disabling shadows or using VFR fixes the issue). These are the results which explain why CFR is the preferred method in my use case.

Video Editor: Shotcut

Working with VFR:

Verdict: way too much effort, time and quality loss, filesize increase and I have to leave the computer running all night.

Screenshots:

Normally this is how it would work by using a CFR video source:

Verdict: Easy peasy and productive.

The thing is that even Davinci Resolve has issues with VFR: https://www.reddit.com/r/davinciresolve/comments/16ve2av/does_dr186_indicate_if_footage_is_variable_frame/

Quote: "Resolve will figure out the video playback fine in most cases, but often it will cause audio desync when cutting. Re-encode as Constant Frame Rate using Handbrake or Shutter Encoder."

As such you have to convert it into CFR for optimal results. The same feedback can be seen over at Shotcut's forum from their community. Same for Kdenlive. So all in all, trying to work with video material in VFR is highly problematic if you want to edit your videos afterwards, but manually converting it from VFR into a video edit friendly format like CFR is way too complicated for longer videos.

dec05eba commented 3 months ago

Just a question. When you record with gpu screen recorder do you have any multimedia application running? if any application is using the nvidia gpu for video decoding/encoding then it will decrease the performance of nvidia encoding in gpu screen recorder, especially as you are hitting the limit of what your gpu is capable of at 4k

Sturmlocke86 commented 3 months ago

Sup mate, nah, the only other thing running besides the game itself is Steam and screen rec. Not sure how much of a performance impact Steam would have here, but according to nvidia-settings my GPU is idling around at zero percent for the video engine after a cold boot. It's a fresh Leap install here too, so there's no bloat or anything running in the background.

I know Steam uses the GPU for some hardware accelerated elements of the gui, but so far no other game is underperforming or anything like that. I could try without Steam just to see what happens. Maybe there is a compat conflict of some sort here?

If the GPU is near it's limit, why doesn't it happen in any other game and why didn't it happen before in eq2? That's what's so strange to me. Something must have changed, but what? I'm on mobile rn and will continue testing in about an hour or so.

dec05eba commented 3 months ago

Ok so, I cant launch everquest II after switching to my nvidia gpu, it worked on my amd gpu. Other games work. Everquest II just quits by itself after pressing play. Did you try the older gpu screen recorder version in everquest? flatpak update --commit=95dfff4968bfd3059a362b01155e2c6034c313515ec15c7e3b66e324f8d8e8ce com.dec05eba.gpu_screen_recorder. Also try recording on a different desktop environment, recording the monitor as well then. Try recording in replay mode as well without saving and see if it still drops to low update fps (to test if there is an issue with the harddrive instead). But other than that I cant really do anything from my side. You said that it just broke by itself but if it doesn't work in old gpu screen recorder version either then something must have happened to your gpu or your desktops configuration. Nobody else has reported this issue either.

Sturmlocke86 commented 3 months ago

Thanks for giving it a go on your Nvidia! It doesn't launch? Are you trying it with Steam + Proton Flatpak or by any other means like vanilla Wine, Lutris etc.? Yeah, I tried the older gpu screen rec version, but it was still doing the same thing, but only for EQ2 of course. All other games can be recorded fine with constant framerate mode. That's why I thought it must be related to something the game is doing now, because if even the older version is showing the same problem, then it probably isn't gpu screen rec. I tried rec the monitor instead of the window, but it does the same thing. I haven't tried rec under a different DE, it's always Gnome here, but I did try different Gnome versions since Ubuntu 23.10 / 24.04 have a more modern iteration of Gnome than what is shipped in Leap. Which DE are you using for the test? I could try something more basic like lxde/lxqt without 3D effects. Ok, will try the replay mode and report back. I actually tried different drives the other day because I have several drives in my pc. The drives I'm using for rec are all NVME SSD, not tried with a mechanical hdd yet. I can test with an hdd though.

Yeah, I'll try all of the steps above and report back asap, but tbh I actually think it might be more related to EQ2 because it started happening right after EQ2 got updated to DX11. It's strange that it's only this game and either switching to variable framerate mode or disabling the in-game shadows fixes the problem completely. Why would disabling shadows fix it? Usually an API change like this will be coupled to some bugs and glitches that get sorted out over time, so it wouldn't surprise me if this is the case. I'll ask around in the EQ2 forums too, maybe someone else has seen this happen.

Whatever the case, thanks a ton for helping out! My donation offer still stands.

Sturmlocke86 commented 3 months ago

Another quick update: I have not tested the above yet, will do so tonight, but I did test streaming to Youtube via gpu screen rec and I noticed a few things:

First test run was alright, but according to the terminal output gpu screen rec switches to constant bitrate mode for Youtube streaming. Is there any way to force variable mode, does yt support that or does it always go to constant mode when streaming? I ask because as you can see in the following video, sometimes in the middle of the stream the fps-rate would tank to 1-fps again: https://www.youtube.com/live/vnb83RKsV1U?si=M8VDFlz6HmOg7Pas&t=3574

I chose to show you the spot at the very end via the link above, which should be at 59:34 minutes, because there you can see how the stream was doing well for the most part, and then the fps-rate tanks all of a sudden and "locks" into that fps-rate. It doesn't go back up again and just stays that way. It did happen earlier, but miraculously it actually recovered and went back up again - not so during the end of the video as seen above though. I stopped the stream at that point. Gameplay was doing fine and did not slow down. That's the same issue that I'm seeing when recording in constant framerate mode only that during recording it happens much faster. For streaming I chose: high quality preset (recommended for streaming), ACC audio, automatic framerate mode (it switched to constant mode automatically) and FLV container. I wasn't able to do MP4 container as it would spit out an error about not being able to stream due to incompatible settings (might have to do with this, will have to retest tonight: "opus audio codec is only supported by .mp4 and .mkv files, falling back to aac instead"). I also tried MKV but the screen remained black during the stream -> I will retest MKV though because it might just have been a hiccup. FLV worked fine though and that's what I used for the video above.

When using the built in gpu screen rec Youtube preset for streaming I wasn't able to stream even after copy and pasting the key that yt provided. It just kept spitting out this error: "GPU Screen Recorder, body: Failed to stream video. There is either an error in your streaming config or your graphics card doesn't support GPU Screen Recorder with the settings you used". So then I switched through all sorts of video and audio settings, nothing worked.

Then I checked the terminal for infos:

Error: Could not open 'rtmp://a.rtmp.youtube.com/live2/': Input/output error
waitpid failed: Keine Kind-Prozesse (= no child process)

Further checking what could be causing this I compared the link above to the link that yt was showing me via yt studio:

rtmp://x.rtmp.youtube.com/live2/

(the backup link uses "y" instead of "x" btw)

Seems like gpu screen rec is trying to use the link with a.rtmp and yt is generating the link with x.rtmp for me via yt studio. Could this be the issue? In order to bypass this I switched to "custom" mode in gpu screen rec and then copy and pasted the streaming url from Youtube directly: rtmp://x.rtmp.youtube.com/live2/KEY

KEY = copy and paste generated yt key.

With that + FLV + ACC audio codec I was able to start streaming, see video above.

Just some feedback before I hit the sheets. Will return later.

Sturmlocke86 commented 3 months ago

Update: Ok, before I go to bed I tested gpu screen rec with IceWM (preinstalled on openSUSE Leap as a fallback option) and LXQT/LXDE and it's the same issue: constant framerate mode makes the fps tank to 1-fps. Variable is buttery smooth, no issues. So it's not the DE (Gnome). Will continue testing later, my left eye is already closed. gn

dec05eba commented 3 months ago

Youtube shows a.rtmp for me, I think it should work for you as well, but I see that there is a bug in the key. The youtube and twitch keys were switched, so it used the wrong key when you tested.. I fixed that now.

Sturmlocke86 commented 3 months ago

Awesome, ty! Is the update out yet? Edit: Ok, update is out now. It was showing 3.7.4 a few seconds ago but it's 3.7.5 now in Gnome Software. I can try it in approx 2 hours and report back. Will try the normal Yt preset in the app and see how it goes. Btw, is it right that streaming to yt will always be in constant bitrate / framerate mode? Is it possible to add a selection between the two in order to do variable instead since which isn't causing any issues in EQ2? Or does yt always default to constant?

Sturmlocke86 commented 3 months ago

Ok, quick update: I tried the replay function in constant framerate mode. Replay without saving: no issues, no frame drops, not even a single fps drop, gameplay buttery smooth at 60fps and 4k reso. Replay after saving video and video playback post recording: 1 fps, unwatchable, same issue as when recording normally. Same here, turning off gpu shadows in-game fixes it. Haven't tried with variable framerate mode yet.

dec05eba commented 3 months ago

Im not sure what you mean, does the update fps stay at ~247 when using replay but then go to 1 fps after you save?

Sturmlocke86 commented 3 months ago

eq2 framerate when recording and replay-min

The framerate starts out fluid at around ~247 but then it suddenly tanks at some point even though standing still. Gameplay is smooth as always though, so the fps issue can not be seen when I'm playing the game. Playing the game EQ2 remains at steady 60fps without any hiccups whatsoever. Only when I save the video and play it back via vlc or mpv is when I can see how the video is performing at 1 fps (same as in the stream that I linked yesterday), which the terminal output can confirm.

sturm@linux:~> flatpak run com.dec05eba.gpu_screen_recorder
Gtk-Message: 00:42:34.092: Failed to load module "canberra-gtk-module"
Gtk-Message: 00:42:34.121: Failed to load module "canberra-gtk-module"
Gtk-Message: 00:42:34.122: Failed to load module "canberra-gtk-module"
window name: EverQuest II (May 10 2024 09:51:45) USER OPTIMIZED
[hevc_nvenc @ 0x55d09f2e6480] ignoring invalid SAR: 0/0
[hevc_nvenc @ 0x55d09f2e6480] ignoring invalid SAR: 0/0
update fps: 241
update fps: 243
update fps: 241
update fps: 242
update fps: 243
update fps: 242
update fps: 242
update fps: 242
update fps: 240
update fps: 242
update fps: 242
update fps: 241
update fps: 243
update fps: 241
update fps: 241
update fps: 243
update fps: 241
update fps: 240
update fps: 241
update fps: 241
update fps: 241
update fps: 243
update fps: 240
update fps: 241
update fps: 242
update fps: 243
update fps: 241
update fps: 240
update fps: 242
update fps: 242
update fps: 242
update fps: 56
update fps: 24
update fps: 13
update fps: 8
update fps: 7
update fps: 5
update fps: 4
update fps: 4
update fps: 3
update fps: 3
update fps: 3
update fps: 3
update fps: 2
update fps: 2
update fps: 2
update fps: 2
update fps: 2
update fps: 2
update fps: 2
update fps: 2
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1
update fps: 1

Now if I go and disable the gpu shadows in-game the fps-rate remains at around ~247 and does not go down to 1-fps according to the terminal logs. The same happens if I switch to variable framerate mode, the fps-rate is steady at ~247 according to the terminal logs even with gpu shadows enabled. So variable + shadows enabled = no problem. Constant + shadows enabled = fps tanks to 1 fps. Constant + shadows disabled = no problem. EQ2 is the only game doing this. I just tried with Cyberpunk 2077 at very high details and 4k reso and the fps-rate is always stable and high, no problem here.

dec05eba commented 3 months ago

Awesome, ty! Is the update out yet? Edit: Ok, update is out now. It was showing 3.7.4 a few seconds ago but it's 3.7.5 now in Gnome Software. I can try it in approx 2 hours and report back. Will try the normal Yt preset in the app and see how it goes. Btw, is it right that streaming to yt will always be in constant bitrate / framerate mode? Is it possible to add a selection between the two in order to do variable instead since which isn't causing any issues in EQ2? Or does yt always default to constant?

I tested with variable framerate and that seems to work on twitch and youtube so i removed the force constant framerate for videos. It will now use constant or variable framerate depending on what the user selects in the settings. It will be available in flathub later (version 3.7.6).

Sturmlocke86 commented 3 months ago

Awesome!

I tested it a bit more and it seems that even variable can actually go down a bit with gpu shadows enabled -> depending on the scene. In some scenes variable always shows stable ~247 with gpu shadows enabled, and in some scenes it can go down to 60-120 (plus, minus), but the recorded video always remains very smooth despite the dip. I saved a video during a scene with gpu shadows enabled and the fps-rate went down to ~70 in variable mode (seen in terminal), but the exported video was still very smooth. The only thing that absolutely tanks to single digit numbers is constant framerate mode with gpu shadows enabled at 4k reso (doing 2k reso helps improve the fps-rate sometimes but not always). I also checked more of my locally recorded videos and I can see that I was able to record in DirectX9 with gpu shadows enabled at 4k60 reso and constant video mode, and it was always buttery smooth. So this further indicates that EQ2 is probably doing something different now after switching to DX11 and that might be causing the additional performance impact / overhead.

I opened up a bug report in EQ2: https://forums.everquest2.com/index.php?threads/gpu-shadows-causing-framerate-issues-when-recording-gameplay.2058/

It seems like other players are also experiencing some hiccups after DirectX11 became the new standard in EQ2: https://forums.everquest2.com/index.php?threads/directx11-stuttering.2060/

So for now having the option in gpu screen rec to either use constant or variable mode for recording and streaming seems to be a good solution to this problem. I actually didn't even know that Yt and Twitch supported variable framerates. Looks like it's been there for a while now: https://www.reddit.com/r/Twitch/comments/13ymnq3/why_does_twitch_encode_videos_as_variable_frame/

So thank you for adding this option! Very much appreciated! Will test both the fixed Youtube streaming preset and the variable streaming mode in a couple of hours.

dec05eba commented 3 months ago

For whatever reason flathub didn't update the package. Maybe because the date I set is 1 day in the future according to USA timezone :man_shrugging:

Sturmlocke86 commented 3 months ago

I was already wondering and checking Gnome Software periodically :) No idea about Flathub, have never uploaded anything there. Do they have an faq for questions like that? No worries though, I'll be here later tonight anyway. On standby for the new version. Will test right away once it's there.

Sturmlocke86 commented 3 months ago

Got the update and will give it a go in a few minutes via a live stream to see how it goes.

Sturmlocke86 commented 3 months ago

Alrighty, typing from my phone now, but glad to report:

I'm actually streaming right now, feel free to check and see how it's doing. You fixed all of the issues! Well done and big thanks for taking care of it all! I wish I could donate for your hard work and dedication, this app deserves every single cent it can get. Thanks to you I can record and stream in high quality now.

dec05eba commented 3 months ago

Thanks for testing. If I find a way to make constant framerate stay at 60 fps even if encoding cant do 60 fps then I'll remove variable framerate and always use that. It's technically possible but no other screen recorder does it and I dont know if ffmpeg/nvenc has any option to do it (since I dont think even ffmpeg can do it when you record directly with ffmpeg).

Sturmlocke86 commented 3 months ago

Yeah, I'm content that it works well with variable. Tbh, I would always like to have the choice between the two, the way it is now basically. It's useful for workaround use cases like mine. You could make constant the default (recommended) but still allow users to use variable for streaming if needed. Having options is good 👍🏻