Vulkan Support branch - Depth Occlusion not working

[RomanCalderon]

Preliminary Checks

• [X] This issue is not a duplicate. Before opening a new issue, please search existing issues.
• [X] This issue is not a question, feature request, or anything other than a bug report directly related to this project.

Description

Hello! I'm raising this issue because I have been checking out your Vulkan support branch and believe I'm experiencing a bug with game objects not being occluded by the Zed camera feed (physical objects closer to the camera). This normally works when used with DX11, however, I'm using another plugin that requires DX12 or Vulkan to work, and would love to get the ZED MR/AR features working along side it.

I've tested the Vulkan support project in isolation to test functionality and can confirm the AR depth occlusion feature isn't working. Other features like the body tracking module works, but AR is what I'm looking for. I've tried in both Built-in RP and URP, using forward rendering as recommended, yielding same issue where game objects simply render on top of the Zed camera feed and no depth-based occlusion occurring.

Using a ZED2i and ZED SDK 4.0

Steps to Reproduce

1. Open or create a new Unity project (I'm using 2022.3.13f1)
2. Bring in the Zed folder (from the Vulkan support branch)
3. Confirm Zed features like BT is working (confirmed on my PC, Windows 10)
4. Run the Simple MR test scene (Assets/ZED/Examples/Simple MR/Scenes/SimpleMR)

Expected Result

Game Objects (spheres, cubes, etc) should clip into physical floor/walls

Actual Result

Occlusion doesn't occur and game objects render completely on top of the Zed camera feed (Frame)

ZED Camera model

ZED2i

Environment

Windows 10
NVIDIA 3060Ti
36 GB ram
Unity 2022.3.13f1
Built-in RP / URP 14.0.9

Anything else?

No response

[Bvallon-sl]

Hi,

Thanks for the report. Indeed, the depth occlusion was not working correctly, my apologies for that.

It should be fixed now. Can you pull the branch and try again?

Thanks again for the feedback.

Best,

[RomanCalderon]

Thanks for taking the time to look into this. Just getting back from the holiday break, will check it out soon!

[RomanCalderon]

Hey @Bvallon-sl

So far so good with your latest commit here. Tested a Unity project (2022.3.16f1) using Vulkan and the Built-in RP with no issues, now getting proper depth occlusion again. Thanks!

I'll want to also verify compatibility with URP and possibly HDRP (although I never use HDRP), but can't give you an exact ETA on that info yet.

Cheers

[Bvallon-sl]

Hi,

Thanks a lot for your feedback.

Let me know if you have any issues with HDRP!

Best, Benjamin Vallon

[RomanCalderon]

Hello,

In testing URP 14.0.9 (latest package version) using the SimpleMR sample scene, both DX11 and Vulkan result in a similar shader error immediately when entering play mode:

(DX11 error message):

Shader error in 'ZED/ZED Forward Lighting URP': undeclared identifier 'zed_xyz' at line 123 (on d3d11)

Compiling Subshader: 0, Pass: StandardLit, Vertex program with _MAIN_LIGHT_SHADOWS_CASCADE _SHADOWS_SOFT Platform defines: SHADER_API_DESKTOP UNITY_ENABLE_DETAIL_NORMALMAP UNITY_ENABLE_REFLECTION_BUFFERS UNITY_LIGHTMAP_FULL_HDR UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BLENDING UNITY_SPECCUBE_BOX_PROJECTION UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS Disabled keywords: NO_DEPTH SHADER_API_GLES30 UNITY_ASTC_NORMALMAP_ENCODING UNITY_COLORSPACE_GAMMA UNITY_FRAMEBUFFER_FETCH_AVAILABLE UNITY_HALF_PRECISION_FRAGMENT_SHADER_REGISTERS UNITY_HARDWARE_TIER1 UNITY_HARDWARE_TIER2 UNITY_HARDWARE_TIER3 UNITY_LIGHTMAP_DLDR_ENCODING UNITY_LIGHTMAP_RGBM_ENCODING UNITY_METAL_SHADOWS_USE_POINT_FILTERING UNITY_NO_DXT5nm UNITY_NO_FULL_STANDARD_SHADER UNITY_NO_SCREENSPACE_SHADOWS UNITY_PBS_USE_BRDF2 UNITY_PBS_USE_BRDF3 UNITY_PRETRANSFORM_TO_DISPLAY_ORIENTATION UNITY_UNIFIED_SHADER_PRECISION_MODEL UNITY_VIRTUAL_TEXTURING _MAIN_LIGHT_SHADOWS _RECEIVE_SHADOWS_OFF

(Vulkan error message):

Shader error in 'ZED/ZED Forward Lighting URP': undeclared identifier 'zed_xyz' at line 123 (on vulkan)

Compiling Subshader: 0, Pass: StandardLit, Vertex program with _MAIN_LIGHT_SHADOWS_CASCADE _SHADOWS_SOFT Platform defines: SHADER_API_DESKTOP UNITY_ENABLE_DETAIL_NORMALMAP UNITY_ENABLE_REFLECTION_BUFFERS UNITY_LIGHTMAP_FULL_HDR UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BLENDING UNITY_SPECCUBE_BOX_PROJECTION UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS Disabled keywords: NO_DEPTH SHADER_API_GLES30 UNITY_ASTC_NORMALMAP_ENCODING UNITY_COLORSPACE_GAMMA UNITY_FRAMEBUFFER_FETCH_AVAILABLE UNITY_HALF_PRECISION_FRAGMENT_SHADER_REGISTERS UNITY_HARDWARE_TIER1 UNITY_HARDWARE_TIER2 UNITY_HARDWARE_TIER3 UNITY_LIGHTMAP_DLDR_ENCODING UNITY_LIGHTMAP_RGBM_ENCODING UNITY_METAL_SHADOWS_USE_POINT_FILTERING UNITY_NO_DXT5nm UNITY_NO_FULL_STANDARD_SHADER UNITY_NO_SCREENSPACE_SHADOWS UNITY_PBS_USE_BRDF2 UNITY_PBS_USE_BRDF3 UNITY_PRETRANSFORM_TO_DISPLAY_ORIENTATION UNITY_UNIFIED_SHADER_PRECISION_MODEL UNITY_VIRTUAL_TEXTURING _MAIN_LIGHT_SHADOWS _RECEIVE_SHADOWS_OFF

[Bvallon-sl]

What version of Unity are you using ?

[RomanCalderon]

For now I'm sticking with 2022.3.16f1

[Bvallon-sl]

I apologize for the inconvenience, It was my fault.

It should be fixed now. Thanks again for the feedback.

[RomanCalderon]

Absolutely, happy to say that did the trick! Confirmed in both DX11 and Vulkan for URP and HDRP, all in 2022.3.16. Thank you for the quick turnaround as well, much appreciated.

I've made my own .unitypackage export of the ZED root folder from a clone of this branch to test with in separate, newly-created projects with their respective graphics/render pipeline configurations, to keep the tests cleanly separated.

Out of curiosity, what do next steps look like for including Vulkan into main/adding an official release package? Moreover, what about DX12 support?

Thanks

[Bvallon-sl]

This will probably be merged for the release of the ZED SDK v4.1, planned for in few weeks.

DX12 support is not planned short term, now that Vulcan is available. Does your project specifically require DX12?

[RomanCalderon]

Sounds good. No, we don't require DX12 as of now, but some other plugins we're experimenting with (regarding custom renderers) require either Vulkan or DX12, so having support in both graphics APIs would be great to have for any future developments with relevant integrations.

[Bvallon-sl]

Ok, that makes sense.

We prioritized Vulcan over DX12 to also support Ubuntu users but if the DX12 is highly requested, we will implement it as well. Thanks again for reaching your feedback, it is really important for us and helps us a lot in building our roadmaps!