Games made with LWJGL3: Performance tanks when being recorded

[SkyAphid]

Version

3.2.1

Platform

Windows x64

JDK

Zulu OpenJDK 17.0.4

Module

LWJGL core, GLFW, OpenAL, OpenGL, STB, JOML

Bug description

When recording footage of our games at resolutions higher than 1080p, performance drops dramatically. For example, our primary project, Robot Farm, goes from 230 FPS to 120 FPS on a basic scene where there's not much going on. In scenes where it would normally be 100 FPS, it can dip to 60 FPS or lower. It appears to only happen on performance intensive software that use OpenGL extensively.

This has also been tested on our older games; We Shall Wake from 2015 on LWJGL2 dropped dramatically when being recorded. As well as "Battledroid" by Puppygames, dipping far below its target of 60 FPS when being recorded at resolutions higher than 1080p. I'm not quite certain this is an LWJGL bug or something else, since as far as I know, LWJGL2 does not share any code with LWJGL3.

Regardless, I felt that it would be appropriate to report, since this could limit the visibility of games made with LWJGL in the future due to the fact that the primary way games are shared/spread in this time are via YouTube/Twitch streaming. If this issue extends past LWJGL into something else, preventative measures could be potentially taken to remedy this.

Builds of all of our games can be supplied for testing. Our build of We Shall Wake running on LWJGL2 can be found publicly here: https://www.nokoriware.com/weshallwake

Recording programs tested:

• Fraps
• Bandicam

Stacktrace or crash log output

No response

[Spasi]

Hey @SkyAphid,

This is completely normal. There's nothing wrong with LWJGL or the recording software. Even if you ignore the bandwidth required and the encoding/storage overhead, reading back the framebuffer is a major synchronization point and will always affect rendering performance. Btw, I wouldn't say the FPS drops you're seeing are "dramatic", they're about 4-6ms per frame, in line with what I would expect when rendering at 1440p or 4K.

There have been many attempts to mitigate the impact (buffering, asynchronous readbacks, etc.) but there's no good software solution to this problem. I guess the best you could do with software is something like Nvidia ShadowPlay, which encodes the framebuffer on the GPU and reduces the amount of data that needs to be read back. Maybe one could even implement a similar solution using Vulkan's H.264/H.265 encoding extensions. However, there's some performance hit even with ShadowPlay.

Fortunately there's an easy (but costly, depending on the output quality/fps required) solution that makes this problem go away and every competitive streamer uses it: a second PC with an HDMI capture card. The GPU of the primary/rendering machine is connected to the capture card via HDMI and the OS is configured to mirror whatever is displayed on the primary monitor. Then the second machine takes care of encoding & streaming. It is the only zero-overhead solution.

[SWinxy]

Drops dramatically more than what would normally be expected? What are the frame times you see before & during capturing, and have you done any profiling?

[SkyAphid]

@Spasi

Fair enough, I'll keep this in mind and give Shadowplay a try. My worry was primarily that it seems to hit games made with Java the hardest, since I'm also getting major stuttering as well. This doesn't seem to occur with games made the "traditional way," but if there's nothing that can be done, I'll do my best to mitigate it.

Regardless, I wanted to report it in case this was something that needed to be diagnosed. My apologies for taking up your time.

Thanks for the quick response fellas!

[httpdigest]

No one (should) ever use software-only solutions like Fraps or Bandicam anymore when recording their gameplay footage... I made all my videos with OBS (with NVENC) and later switched to GeForce Experience to record all my videos at frametimes of less than a millisecond (> 1000 FPS) on 1440p. The overhead of NVENC-based solutions like GeForce Experience (previously ShadowPlay) or the encoder in OBS is so absolutely minuscule. But, like @Spasi said, if you want to have zero overhead, you need a separate PC with a capture card.

So, if you have an Nvidia card with a hardware video encoder engine, then by all means, use that with either OBS or GeForce Experience.

Also: If you choose OBS make sure to start it as the PC's Administrator account, as Windows 10+ display driver scheduler will then give it way more GPU compute time when it needs it, as otherwise (when run under a non-privileged Windows user) you can still get hitches in the encoded video (but not in the live gameplay) that you will then see in the recorded video later.