search

Unity-Technologies / UnityRenderStreaming

Streaming server for Unity
Other
1.3k stars 352 forks source link

[BUG]: After upgrading to 3.1.0 exp7, the 4K video stream is too laggy #941

Open ViKeei opened 1 year ago

ViKeei commented 1 year ago

Package version

3.1.0-exp.7

Environment

* OS:Windows 10
* Unity version:Unity2022.3.1
* Graphics API:DirectX11
* Browser:Microsoft Edge 116

Steps To Reproduce

After upgrading to 3.1.0 exp7, the 4K video stream is too laggy, the screen of 3.1.0 exp6 is very smooth. with the same scene : 3.1.0 exp7 : 4K 25FPS H264 3.1.0 exp6 : 4K 37FPS H264 Another issue is that Unity crashes when switching Unity Render Streaming versions and immediately connect to web video streaming.

Current Behavior

No response

Expected Behavior

No response

Anything else?

Video_20230818110332_edit.zip

karasusan commented 1 year ago

Hi, I think this fix is related your issue. We already merged it. https://github.com/Unity-Technologies/com.unity.webrtc/pull/962

karasusan commented 1 year ago

And as a workaround, the crush might not be occurred using DX12.

ViKeei commented 1 year ago

Things seem to be getting worse, but the 3.1.0 exp6 version is still fine. If opened in default mode, unity will crash directly.There are two types of crashes in the video. Video_20230818143031.zip

karasusan commented 1 year ago

@ViKeei Let me check this issue. Please tell me the graphics driver version just in case. Also, is this crash relating the 4K resolution? How about FHD resolution?

ViKeei commented 1 year ago

NVIDIA RTX 3060Ti, Game Ready 531.18 QHD and 4K UHD both crash, FHD not.

karasusan commented 1 year ago

@ViKeei Thanks, we're going to check the 4K resolution again.

karasusan commented 1 year ago

@ViKeei Sorry, I have a question, is this issue reproduced when using DX12 as a graphics device instead of DX11?

ViKeei commented 1 year ago

@karasusan At your suggestion, subsequent tests and replies have been done in DX12. Compared to exp6's version, exp7's version running under DX11 only results in 4K frame rate drops and choppy graphics.

karasusan commented 1 year ago

WRS-495

karasusan commented 1 year ago

@ViKeei Thanks, I want to know this fix is related your issue or not. Can you try it if I provide a package which contains this fix? https://github.com/Unity-Technologies/com.unity.webrtc/pull/962

ViKeei commented 1 year ago

@karasusan OK, no problem.

karasusan commented 1 year ago

@ViKeei https://drive.google.com/file/d/19COIA7sdb5jSd3Rijt9VDVy91BKyXZrQ/view?usp=drive_link

ViKeei commented 1 year ago

I tested it on both of my computers and both crashed. One is Windows10, one is Windows11.Test in Unity2022.3.1，Unity2023.1.8. In Unity2022, it crashes the first time it runs, and after restarting, the web page screen appears but the screen doesn't move and the bitrate is zero. In Unity2023, it crashes every time.

karasusan commented 1 year ago

@ViKeei OK, this crash must be a new issue, I put a zip file contains the native plugin built for debug. Please try it and send me a crash log. https://drive.google.com/file/d/1MqZtJgUCeRNEXm4MbgJaneEfm-OoMTWV/view?usp=drive_link

ViKeei commented 1 year ago

@karasusan Editor.log

ViKeei commented 1 year ago

this is from Unity 2023.1.8 Crashes.zip 20230824210717

karasusan commented 1 year ago

@ViKeei I am trying to replicate your issue on my windows machine, but I couldn't yet. The assertion means that ID3D12Resource::Map is failed in D3D12GraphicsDevice.cpp.

I don't think there are differences between Unity Editor and one of China version.

ViKeei commented 1 year ago

@karasusan I can't say for sure if it's an issue with the Chinese version, but I'm experiencing crashes in the non-Chinese version as well. I created multiple brand new projects, but the crashes didn't happen in every project, and in some of them I did a Unity render streaming version switch. But for sure the exp6 version works fine in all tests. Maybe it's just me.

Framerate comparison for the same scene: Dingtalk_20230825134503 Dingtalk_20230825131400

This is a crash from Unity2023.1.8f1： Editor.log

karasusan commented 1 year ago

@ViKeei Thank you for the clarification. Is it possible to identify the difference of the crash project and the not crash? I would be good if I could check the minimum project for replicate the crash.

lijunhe1991 commented 1 year ago

Is there a solution to this problem? I have the same problem here.

karasusan commented 1 year ago

@lijunhe1991 The performance issue with DX11 is replicated on your environment? I'd like to know more details like PC specs and driver version.

lijunhe1991 commented 1 year ago

@karasusan

The default frame rate is 30 frames, the 1K frame rate is normal, and the 4K frame rate is only about 10 frames and unstable.

At 2023-09-01 10:08:58, "Kazuki Matsumoto" @.***> wrote:

@lijunhe1991 The performance issue with DX11 is replicated on your environment? I'd like to know more details like PC specs and driver version.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

lijunhe1991 commented 11 months ago

@karasusan Is there a solution to this problem?

karasusan commented 11 months ago

@lijunhe1991 @ViKeei

It looks no problem in my environment.

https://github.com/Unity-Technologies/UnityRenderStreaming/assets/1132081/5f297c7c-afc0-4e6b-87e6-96c493fece63

This is my system info.

NVIDIA System Information report created on: 09/15/2023 18:05:50 System name: LAPTOP-HBGEGGU7

[Display] Operating System: Windows 10 Pro, 64-bit DirectX version: 12.0 GPU processor: Quadro RTX 5000 Driver version: 529.08 Driver Type: DCH Direct3D feature level: 12_1 CUDA Cores: 3072 Core clock: 1350 MHz Memory data rate: 12.00 Gbps Memory interface: 256-bit Memory bandwidth: 384.06 GB/s Total available graphics memory: 32619 MB Dedicated video memory: 16384 MB GDDR6 System video memory: 0 MB Shared system memory: 16235 MB Video BIOS version: 90.04.7D.00.17 IRQ: Not used Bus: PCI Express x16 Gen3 Device Id: 10DE 1EB5 229717AA Part Number: 4914 0010

[Components]

nvui.dll 8.17.15.2908 NVIDIA User Experience Driver Component nvxdplcy.dll 8.17.15.2908 NVIDIA User Experience Driver Component nvxdbat.dll 8.17.15.2908 NVIDIA User Experience Driver Component nvxdapix.dll 8.17.15.2908 NVIDIA User Experience Driver Component NVCPL.DLL 8.17.15.2908 NVIDIA User Experience Driver Component nvCplUIR.dll 8.1.940.0 NVIDIA Control Panel nvCplUI.exe 8.1.940.0 NVIDIA Control Panel nvWSSR.dll 31.0.15.2908 NVIDIA Workstation Server nvWSS.dll 31.0.15.2908 NVIDIA Workstation Server nvViTvSR.dll 31.0.15.2908 NVIDIA Video Server nvViTvS.dll 31.0.15.2908 NVIDIA Video Server nvLicensingS.dll 6.14.15.2908 NVIDIA Licensing Server nvDevToolSR.dll 31.0.15.2908 NVIDIA Licensing Server nvDevToolS.dll 31.0.15.2908 NVIDIA 3D Settings Server nvDispSR.dll 31.0.15.2908 NVIDIA Display Server nvDispS.dll 31.0.15.2908 NVIDIA Display Server PhysX 09.21.0713 NVIDIA PhysX NVCUDA64.DLL 31.0.15.2908 NVIDIA CUDA 12.0.151 driver nvGameSR.dll 31.0.15.2908 NVIDIA 3D Settings Server nvGameS.dll 31.0.15.2908 NVIDIA 3D Settings Server

lijunhe1991 commented 11 months ago

@karasusan H264 frame rate is stable and normal，other codec frame rate is very low.

karasusan commented 11 months ago

@lijunhe199 Generally, encoding 4K video with software codec is verh high load. I recommend you to use H264 codec.

lijunhe1991 commented 11 months ago

Is there any plan to support GPU encoding for other encoding formats?

karasusan commented 11 months ago

@lijunhe1991 I think AV1 NVIDIA codec should be supported. Also oneVPL codec support is useful for many customers. However, I can't promise when we support them.