Virtual Canvas camera feed shows for a second then goes black?

ROBYER1 commented 6 months ago

Preliminary Checks

[X] This issue is not a duplicate. Before opening a new issue, please search existing issues.
[X] This issue is not a question, feature request, or anything other than a bug report directly related to this project.

Description

Started a new Unity project using URP in Unity 2023.1.19f1, imported the SDK and started running the body tracking scene. Once the camera spinner disappears, the camera feed shows for a second then turns black. This is happening with two identical Zed 2i cameras on two different PCs here - one is a HP Zbook laptop on Win11 and the other is a Windows 10 desktop PC, both running RTX 2070 GPUs

Steps to Reproduce

Install Zed sdk requirments
Import the 4.0.7 sdk to a new Unity project using URP in Unity 2023.1.19f1
Try running the body tracking scene and see if the camera image renders after a second or two

Expected Result

Virtual Canvas camera image should show the camera feed

Actual Result

Virtual canvas camera image doesn't show the camera feed, body tracking does work however. See the issue here in a gif, camera goes black after a second loading

ZED Camera model

ZED2i

Environment

OS: Win 10 and 11
Nvidia RTX 2070 GPU laptop and PC sized
Intel Core i9 vPro CPU
CPU performance of app is fine
Unity 2023.1.19f1
Zed SDK version: https://github.com/stereolabs/zed-unity/releases/tag/v4.0.7

Anything else?

No response

SLJLMacarit commented 6 months ago

Hello @ROBYER1, Thanks for reaching out!

I've got some questions:

Did you upgrade the ZED Package to URP and did it go well?
Do you get any errors in the console?
Does it work correctly in a non-URP project?
Does the Depth Viewer tool work correctly?
In the second where the camera feed is visible, can you check if the image changes, or if it's a fixed image?

ROBYER1 commented 6 months ago

@SLJLMacarit This happens in separate projects on separate PCs but the setup was the same:

Did you upgrade the ZED Package to URP and did it go well? Yes, upgraded to URP and no errors
Do you get any errors in the console? No errors in the console when running
Does it work correctly in a non-URP project? I can test this but we need it to work with URP
Does the Depth Viewer tool work correctly? Yes, in the Zed viewer we can see the camera feeds just fine for colour and depth
In the second where the camera feed is visible, can you check if the image changes, or if it's a fixed image? The camera feed that shows for a second is a live camera feed at that time, it shows anything moving in the room

Here is a fresh copy of a test project made from scratch, same issue happens there in it with the Zed 2i, Unity2023.1.19f1 https://drive.google.com/file/d/1jGSEm5H43yZzbM6CfEIPtmNSowGFaoBc/view?usp=sharing

RomanCalderon commented 6 months ago

I am experiencing this issue as well when using the body tracking module, but only when the MirrorMode property under Avatar Control is enabled. Without mirror mode, the camera feed remains visible. Using Unity 2022.3.13f1 and URP v14.0.9

ROBYER1 commented 6 months ago

In our scenario, mirror mode remains unticked/off and we don't intend to use mirror mode

SLJLMacarit commented 6 months ago

@RomanCalderon @ROBYER1 I haven't been able to take a look yet, I'll do it asap, but from what you're saying I understand that it only happens with the Body Tracking scene. Is that the case?

RomanCalderon commented 6 months ago

Hey @SLJLMacarit thanks for looking into this. It happens for me when using the body tracking module component, both within the example scene provided in the ZED Unity package and when I set up a new scene entirely, using the module + ZED rig mono.

All functionality seems to work ok, but when mirror mode is enabled, the feed goes blank after a few frames.

ROBYER1 commented 6 months ago

@RomanCalderon @ROBYER1 I haven't been able to take a look yet, I'll do it asap, but from what you're saying I understand that it only happens with the Body Tracking scene. Is that the case?

Yes, this is only happening in the body tracking scene for us, and is happening without mirror mode enabled. Other test scenes work fine but I noticed the body tracking scene uses a mono camera rig.

SLJLMacarit commented 6 months ago

@ROBYER1 I finally managed to take a look. It seems that the way the mirror is done doesn't work in URP. You can either disable it or modify the code to flip the X scale value of the "ZEDview" canvas in the scene instead of the "Frame" object.

I'll log it for an update, thanks for the report, and happy New Year!

ROBYER1 commented 6 months ago

What if this issue is happening when I'm not using Mirror mode? I haven't been using Mirror mode at all although I know other people mentioned it causing issues also.

Can you please confirm if there is a fix for this without any use of mirror mode like in my original post on this issue?

SLJLMacarit commented 6 months ago

I haven't managed to reproduce the issue without mirror mode on my side.

To totally rule it out, can you send how the "Frame" object's Transform looks in the Inspector at runtime, both with mirror mode enabled and not? It's located under ZED_Rig_Mono > Camera_Left > Frame.

(Note: you have to start the scene with mirror mode enabled or not to change the camera view, it doesn't update at runtime)

On my side for example, the X scale is negative, and in URP it produces a texture stretched to 0 scale.

github-actions[bot] commented 4 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days

stereolabs / zed-unity