We are developing an Open-source Low-latency Game Streaming client

[Opensourcestreamingclient]

Hello everyone,

We are developing an Open-source Low-latency Game Streaming client based on Sunshine. https://open-stream.net

We are facing this strange issue: We manged to get Open-Stream working with our AMD 5700xt systems. However, when we upgraded the drivers from Adrenalin 2020 Edition 20.5.1 to Adrenalin 2020 Edition 20.12.1, there is huge lag in the display with nearly 100% GPU usage (or any version in between). It does not matter if we implement CPU encoding or GPU AMF encoding.

We can see the GPU at less than 10% constantly and when we just rapidly move the mouse cursor, the GPU goes to almost 100% immediately. Is there something we are missing? The same driver versions work perfectly with our AMD RX 480 systems. Also no issues to report with any Nvidia based GPU systems (including our RTX 2080's).

We have already tried upgrading to ffmpeg 4.3.1 along with the h.264 and h.265 codec library's.

Details:

Windows 10 AMD RX 5700xt Adrenalin 2020 Edition 20.121 Processor: AMD 3900x Open-stream version: openstream_alpha_2312.1

Please keep in mind that we are far from being experts at this. There has been a lot of trial-and-error to get to this stage.

We would be happy to offer test accounts for anyone interested in debugging this issue along with a small bounty of $500 Canadian for the first successful solution.

[MikhailAMD]

Few things to note and to check:

• HW encoder performance and GPU load from it is constant. From your description it looks like your frame submission rate is variable, your solution submits more frames per second when user moves the mouse. This would produce more outputs and as the result, more decode jobs on the client side. You may want to check your capture code for this.
• HW encoder is a library, it does not keep any timing/pacing. The frame rate you provided is used for bitrate control only. So timing is app responsibility.
• Check your encoder parameters. you should set AMF_VIDEO_ENCODER_LOWLATENCY_MODE = true.
• Check that VBV buffer is no more that 1-2 frames: bitrate/framerate.
• Check for return codes when you set parameters, RX 5xxxx encoder is different from RX 4xx and some restrictions may apply to your parameter.
• Check debug output for warnings from AMF.
• You may want to add some output bitrate statistics to your code to see what is going on in real-time

[Opensourcestreamingclient]

Thank you for the quick reply. You will find my comments below.

• No issues to report on the client side. https://moonlight-stream.org/ is used on the client end. We have also experimented with CBR and the GPU goes to nearly 100% without any movement . Again only with AMD driver versions 20.7.1 or higher. No issues to report with Adrenalin 2020 Edition 20.5.1 or older.

• We will go over the timing of the APP. With that said no issues to report with Adrenalin 2020 Edition 20.5.1. So maybe something fundamentally has changed since that software revision.

• AMF Encoder is set currently to ultralowlatency mode (we have also tried lowlatency).

• The default VBF buffer is set to about 1 or 4 frames (1Mbps or 1000 Kilobit).

• We will check the return codes

• We will check the debug output for warnings from AMF as well as real-time statistics

Please note that this bug is also affecting CPU encoding (h.264 and h.265). For what ever reason the GPU goes to 100% while the CPU is barely used.

Any other suggestions would be welcomed.

[MikhailAMD]

You lost me:

• If this bug is visible with CPU encoder, how come AMF is involved?
• You can always record a short GPUVIEW log (5-10 sec) and see who is using the GPU in the bad case. If you have troubles to understand it, you can ZIP and share the log with me.

[Opensourcestreamingclient]

This bug is visible with both AMD AMF and CPU encoding with any AMD driver version newer than 20.5.1. Maybe we are using a call function that is no longer available in the latest AMD driver versions.

I will run the GPUVIEW log and share it with you.

[Opensourcestreamingclient]

GPU AMF H.264 GPUVIEW log:

https://open-stream.net/wp-content/uploads/2020/12/Merged-GPU-AMF-Encoding-Adrenalin-2020-Edition-20.12.1.etl_.tar.gz https://open-stream.net/wp-content/uploads/2020/12/log.cmd-GPU-AMF-Encoding-Adrenalin-2020-Edition-20.12.1..txt

CPU Encoding H.264 GPUVIEW log: https://open-stream.net/wp-content/uploads/2020/12/Merged-CPU-Encoding-Adrenalin-2020-Edition-20.12.1.etl_.tar.gz https://open-stream.net/wp-content/uploads/2020/12/log.cmd-CPU-Encoding-Adrenalin-2020-Edition-20.12.1..txt

When moving the mouse cursor, the 3D part of GPU goes almost immediately to 100%. With Adrenalin-2020-Edition-20.5 the 3D part of the GPU in the task manager averages around 10-15% while the Video encoder can reach as high as 50%. With Adrenalin-2020-Edition-20.12.1 the Video encoder stays at 0%. This in turn causes massive stutter and a drop in the streaming frame rates.

I find it strange that it's affecting the CPU encoding as well. This leads me to believe it might be unrelated to AMF.

[MikhailAMD]

Thanks for the logs. They explain a lot. The common problem with CPU and with GPU encoder is that the application keeps acquired texture/surface from DD capture for a long time and do not release it. This blocks GPU other operations and DWM present and therefore the next capture. Please check AMF DVR sample and example of Desktop Duplication API capture usage: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/blob/master/amf/public/src/components/DisplayCapture/DDAPISource.cpp

See the attached screenshots with explanations: CPU:

GPU

[Opensourcestreamingclient]

Your feedback really helped and I can't thank you enough. Please let us know to where this bounty will be sent.

[MikhailAMD]

I work for AMD and the fact that you are adding support for AMD GPUs and AMF is the best reward. Keep doing this. Also you may want to update us on your progress, publicly or privately.