Not working for me - Githubissues

ASchneiderBR commented 2 years ago

Hello @kounoike, thanks for taking the time to create this tool :)

Not sure what I'm doing wrong: after adding on both video effects and filters, my webcam image simple goes blank, 100% transparent.

I'm trying to use it on a webcam source, 720p30.

Also, I love the idea of using this kind of plugin but since you are using this effect on two different places of the same source, this makes unusable on other kind of sources, like a browser source. I was looking for a solution like this one to use with VDO.Ninja but it seems that would not be possible, since it doesn't have the video effects part, inside filters:

Do you think that would be possible someday?

Thanks!

kounoike commented 2 years ago

Thanks to using this plugin and posting first issue!

hmm... Could you show me your OBS log? (Help menu -> Log Files -> View Current Log) and how about mask image? Filters -> Effect Filters -> Virtual Background Renderer -> Rendering mode set to "render only mask image". White? or black? or anything else?

Yeah, of cause, I like more simple filter settings. But this is restriction by obs-studio's plugin calling interface. Only Video filter passes frame pixel data to C/C++ plugin. and Only Effect Filter can render image with alpha-channel. So, two-filter architecture is required. And due to this reason, only Video Capture Device and Media Source (or some other plugin's source) can use this plugin.

kounoike commented 2 years ago

@ASchneiderBR sorry, forgotten mention

ASchneiderBR commented 2 years ago

Hey there!

Here's a log file. When I enable the first filter, it starts spamming my log with the error device_texture_create (D3D11): Failed to create 2D texture (80070057), then after the second one is turned on, OBS crashes now...

2021-10-06 13-44-24.txt

Too bad this plugin would be usable by a browser source, that would be the main feature for me :)

Thanks a lot!

kounoike commented 2 years ago

Thanks for providing log!

Oh, it curious for me.

13:44:26.304: [Virtual BG detector] Can't create Session error: Exception during initialization: D:\a_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\FusedGraphKernel.cpp(64)\onnxruntime.dll!00007FFD264A71AA: (caller: 00007FFD264A661C) Exception(1) tid(6dac) 80004002 N縊 h・suporte para esta interface

it shows failed to create DirectML Execution Provider. by https://github.com/microsoft/DirectML , DirectML supports your GPU, and it doesn't require any additional software installations. hmm... is onnxruntime exactly same version as provided in this plugin?

13:45:05.665: error: Picture size 0x0 is invalid 13:45:05.665: video_scaler_create: av_image_alloc failed: -22 13:45:05.665: [Virtual BG detector] Can't create video_scaler_create -2

This shows, your camera outputs width: 0, height: 0 pictures...

And, Creating 0x0 texture by Renderer is bug. It will be fixed by next release.

So, I'm planning to release CUDA execution provider with this plugin. I think it resolves first issue. but, it is not helpful for second issue...

I want to help you. But that is a limitation of OBS-Studio.

LawrenceWence commented 2 years ago

Hi, not sure if this helps, I see exactly the same behaviour (NVIDIA GeForce GTX 1050 Ti), with quite current driver.

09:45:44.853: error: Picture size 0x0 is invalid 09:45:44.854: video_scaler_create: av_image_alloc failed: -22 09:45:44.855: [Virtual BG detector] Can't create video_scaler_create -2

Videosource is a Logitech Webcam

Later on we also see: 09:45:48.654: device_texture_create (D3D11): Failed to create 2D texture (80070057) 09:45:48.654: device_texture_create (D3D11): Failed to create 2D texture (80070057)

I doublechecked that there's only the onnx-runtime in the obs folder that came with the latest version of your plugin (version 1.9.21.921)

ASchneiderBR commented 2 years ago

is onnxruntime exactly same version as provided in this plugin?

Hello! Yes, same files, unziped to a brand new, test OBS portable instance.

Thanks for the help :)

lucasbfr commented 2 years ago

Just to confirm the same issue here and thank you for your efforts, because this looks like a promising plugin!

18:19:06.662: [DShow Device: 'Razer'] settings updated: 18:19:06.662: video device: Périphérique vidéo USB 18:19:06.662: video path: \?\usb#vid_1532&pid_0e03&mi_00#6&d3abc98&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global 18:19:06.662: resolution: 640x480 18:19:06.662: flip: 0 18:19:06.662: fps: 30.00 (interval: 333333) 18:19:06.662: format: YUY2 ... 18:19:24.785: User switched to scene 'Camera Razer' 18:19:24.785: error: Picture size 0x0 is invalid 18:19:24.785: video_scaler_create: av_image_alloc failed: -22 18:19:24.785: [Virtual BG detector] Can't create video_scaler_create -2 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.819: error: Picture size 0x0 is invalid

DarkWolfGg commented 2 years ago

Just to confirm the same issue here and thank you for your efforts, because this looks like a promising plugin! x2

18:19:06.662: [DShow Device: 'Razer'] settings updated: 18:19:06.662: video device: Périphérique vidéo USB 18:19:06.662: video path: \?\usb#vid_1532&pid_0e03&mi_00#6&d3abc98&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global 18:19:06.662: resolution: 640x480 18:19:06.662: flip: 0 18:19:06.662: fps: 30.00 (interval: 333333) 18:19:06.662: format: YUY2 ... 18:19:24.785: User switched to scene 'Camera Razer' 18:19:24.785: error: Picture size 0x0 is invalid 18:19:24.785: video_scaler_create: av_image_alloc failed: -22 18:19:24.785: [Virtual BG detector] Can't create video_scaler_create -2 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.786: device_texture_create (D3D11): Failed to create 2D texture (80070057) 18:19:24.819: error: Picture size 0x0 is invalid

fgolman commented 2 years ago

I'm having same issues as above. Running on a Lenovo T480 laptop. I've tried it with integrated camera, USB camera and NDI camera, all with same negative results.

Please advise as your solution looks very promising!

Thanks.

simonbunge commented 2 years ago

I believe I have figured out the root cause of this one...

This error isn't caused by something being wrong with the actual video source, but the rather the tensor side of the scaler initialization:

13:44:59.431: error:   Picture size 0x0 is invalid
13:44:59.432: video_scaler_create: av_image_alloc failed: -22
13:44:59.432: [Virtual BG detector] Can't create video_scaler_create -2

This is caused by the failure to initialise the graph earlier on:

13:44:26.304: [Virtual BG detector] Can't create Session error: Exception during initialization: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\FusedGraphKernel.cpp(64)\onnxruntime.dll!00007FFD264A71AA: (caller: 00007FFD264A661C) Exception(1) tid(6dac) 80004002 Não há suporte para esta interface

This, in turn, is due to the DirectML version shipping with Windows not (necessarily) being the latest in terms of supporting the full opset of the model used.

I was able to reproduce this error on Windows 10 Build 19042.1288). Deploying the DLL from the current DirectML v1.7.0 from here next to the generated plugin DLL (and recompiling against the corresponding header for that matter) fixed it! So to avoid such issues more generally/ permanently, you might actually want to package up DirectML as you're already doing with ONNX Runtime.

kounoike commented 2 years ago

I see that the earlier version of Windows 10 did not include DirectML. That makes sense. I also reproduced it on my Win10/1909 machine and fix it by bundle DirectML DLL.

This fix and some error check fix is released as v1.1.0-pre. Could you try this release? @simonbunge @ASchneiderBR

simonbunge commented 2 years ago

I see that the earlier version of Windows 10 did not include DirectML. That makes sense. I also reproduced it on my Win10/1909 machine and fix it by bundle DirectML DLL.

This fix and some error check fix is released as v1.1.0-pre. Could you try this release? @simonbunge @ASchneiderBR

Tested against a 'fresh' portable version of OBS 27.1.3 and works beautifully out of the box! 👍🏻 🏆

ASchneiderBR commented 2 years ago

Hello @kounoike, I can confirm that your plugin is now working correctly :) Thanks for that.

kounoike commented 2 years ago

@simonbunge @ASchneiderBR thanks for reporting! I will release v1.1.0.

kounoike / obs-virtualbg

Not working for me #31