zappar-xr / universal-ar-unity

57 stars 13 forks source link

Camera background not rendering #23

Closed GSOsborne closed 9 months ago

GSOsborne commented 1 year ago

The Camera is clearly tracking, and the occlusion mesh is working just fine, but the background that my camera should be rendering isn't showing up in the Game window. This is odd because we were able to get all of this working previously, perhaps there was an update that changed how the camera is rendered?

To Reproduce Steps to reproduce the behavior:

  1. Used UAR version 3.2.3
  2. With Unity version 2022.3.4 using SRP and build platform as PC (Windows/Mac/Linux)
  3. Create Zappar Camera using the Zappar toolbar for Zappar > Camera > Front Facing Camera
  4. Zappar > Face Tracking > Face Tracking Target
  5. Created child cube of Face Tracking Anchor
  6. See that while the cube is rendered and even gets occluded by the face mesh, you do not see the output of the camera. So it's clearly doing some kind of tracking, but the texture output is failing.

Expected behavior I expect to see the output of my camera in the Game window.

Screenshots ZapparError

https://github.com/zappar-xr/universal-ar-unity/assets/44345509/7f43753e-6c3e-4d19-aab6-dd8fb06824bc

We're using Zappar AR as a way to teach kids to make their own Face Filters in Unity. We've previously run this course with success, and in fact we have a working template project that we provide them. It's just that setting up the project with a fresh Unity project seems to not be working. I suspect there was some kind of update that broke it. My next steps will be to try downgrading the UAR package version and seeing if that fixes things.

GSOsborne commented 1 year ago

I've checked, and the previous version that still works properly in the editor was 3.2.0, in Unity 2020.3.13f1. I wish Unity 2022.3.4 package manager allowed us an easy downgrade.

nfynt-zap commented 1 year ago

Hello @GSOsborne

From your Camera inspector I notice that you haven't updated UAR SDK to work with Unity SRP. Please check the following section to understand the flow: https://github.com/zappar-xr/universal-ar-unity#supporting-unity-scriptable-render-pipeline

Further, it seems from Unity 2022 LTS on PC the default Color space has been changed from Gamma to Linear, which gives the D3D11 errors for external camera textures. A quick way to work around this would be to switch to Gamma mode after enabling the SRP, you can find this setting under Project Settings/Player/Standalone/Other Settings/Rendering/Color Space. This should fix any black camera frames in editor play mode.

I am yet to throughly test our SDK with latest Unity 2022 LTS version, so will update this thread when the new fixes are live.

Also, if you're creating any templates using our git package you can use specific version in the git URL that you're happy with. You can do so by using https://github.com/zappar-xr/universal-ar-unity.git#VERSION_NUMBER and replace VERSION_NUMBER with your desired version of the package, i.e. https://github.com/zappar-xr/universal-ar-unity.git#3.2.4 for the current release.

GSOsborne commented 1 year ago

https://github.com/zappar-xr/universal-ar-unity/assets/44345509/bd3db1e8-f81a-41b2-9ec4-0a6ed5921d6a

Thought I'd update you. Made sure to update to SRP. Then tried to switch Color space to Gamma, but after these changes same problem persisted.

This is still using Zappar 3.2.3

nfynt-zap commented 1 year ago

Doesn't look like you've updated it, otherwise you would be getting an option to change it to StandardPipeline (Built-in one). Another place you can check whether the SRP is enable, is in UAR Settings under Project Settings.

After that, you would want to delete the existing ZapparCamera from Scene and place a new one from Zappar menu. Or you can try updating the existing scene (which was created in a non-srp mode) for SRP by using menu option Editor/Update Zappar Scene For SRP.

GSOsborne commented 1 year ago

https://github.com/zappar-xr/universal-ar-unity/assets/44345509/7f5bae39-4922-48ec-8ff6-945f8b75f80d

Thank you for your help! I started in a brand new project, imported the Universal Render Pipeline package, imported UAR 3.2.3, made sure the Color Space was set to Gamma, and everything worked! Much appreciated for your help.