search

RenderHeads / UnityPlugin-AVProVideo

AVPro Video is a multi-platform Unity plugin for advanced video playback
https://www.renderheads.com/products/avpro-video/
233 stars 28 forks source link

360 stereo video not stereo anymore with 2.7.1 #1443

Closed RuudvanReenen closed 1 year ago

RuudvanReenen commented 1 year ago

Please DO NOT LINK / ATTACH YOUR PROJECT FILES HERE

Describe the issue With AVPro 2.6.7 and 2.7.1 stereo 360 video (top/bottom) only shows the left (red when tinting is on) image for both eyes, both in Editor (with Android set as target) and when built (to Pico Neo 3 Pro). Using Multipass rendering (PicoMobileSDK, non XR), with a PVRCameraRig with separate cameras for the left and right eye. There's an Update Multi Pass Stereo component on the BothEye (center) camera. Exact same scene works fine with AVPro 2.2.2f1.

Fallback Media Hints Stereo Packing is set to Top/Bottom Apply To Mesh Automatic Stereo Packing is disabled Override Stereo Packing is set to Top/Bottom The sphere mesh material is using the AVProVideo/VR/InsideSphere Unlit shader Stereo mode of the material set to Top_Bottom, Force Eye Mode None, Layout None

Your Setup (please complete the following information):

Chris-RH commented 1 year ago

Hi @RuudvanReenen,

Are you using SRP, URP or HDRP? Have you tried using VR-InsideSphere-Transparent? Is AndroidOES enabled? Have you tried going back to basics with a new project with just our 360 demo scene and building up from there? Is this the one you are using?https://sdk.picovr.com/docs/sdk/en/chapter_one.html

Chris-RH commented 1 year ago

Is this still a problem?

RuudvanReenen commented 1 year ago

Yes, still an issue, I've been using 2.2.2 instead.

Chris-RH commented 1 year ago

How is it in a new project just with AVPro Video and the demo scene?

RuudvanReenen commented 1 year ago

Using Pico UnitySDK version 2.8.11.5. Confirmed both eyes being red with a new project on 2018.4.36f1 and the 360 demo scene, with the camera replaced by a PVRCameraRig with separate cameras for left, right and both eyes. I'll send a link to the project via email.

RuudvanReenen commented 1 year ago

Same issue with AVPro 2.7.2

Chris-RH commented 1 year ago

Using your scene with the Pico SDK, I updated from 2018.4.36f1 to 2021.3.16f1 and set up XR Plug-in Management. Selected XRCore and changed it to Multipass, and it works. Are you able to use a newer version of Unity?

RuudvanReenen commented 1 year ago

Hi Chris, with 2021 and XR there are no issues indeed. But I'm stuck with 2018.4 for some apps due to a major rewrite requirement to switch from UNET when using more recent Unity versions. And AVPro 2.2.2 works with 2018.4, but stereo display doesn't work anymore for 2.7.1 and 2.7.2.

Chris-RH commented 1 year ago

Have you tried copying the shaders from 2.2.2?

RuudvanReenen commented 1 year ago

Tried with the example project, no difference

Ste-RH commented 1 year ago

Can you not update to any major Unity version? Maybe 2019?

The problem will be rooted in lack of available shader level matrices to determine which eye is being rendered in each pass. These have been seemingly randomly available over time in Unity, with them relying on VR SDKs in the main (until more recent versions of Unity). There is essentially no solution, as the variables are just not available. Updating Unity is your only option.

As a side note, Unity do not even let us build packages with 2018 any more! I understand legacy projects and the risks of changing Unity versions, but if this is a blocker, you have a choice to make. No stereo videos, or update. I would try Unity 2019.

RuudvanReenen commented 1 year ago

UNET was dropped after 2018 and this app heavily uses UNET for networking. Rewriting all networking to use Mirror or something similar is not feasible for this one. Weird that AVPro 2.2.2 does work though, I'll just stick with that version for this app.

Ste-RH commented 1 year ago

Are you sure 2.2.2 works? Have you enabled the debug eye tinting? (should get red in one eye and green in the other no matter what head orientation you have)

The only component parts that are the shaders and the UpdateMultiPassStereo script. They will have indeed changed since v2.2.2, but I can't see how what used to be in them would work then and not now.

RuudvanReenen commented 1 year ago

Yes, 2.2.2 works

Ste-RH commented 1 year ago

Try replacing \Assets\AVProVideo\Runtime\Shaders\AVProVideo.cginc in whatever latest version you are using (2.7.1?) with this one. Can't promise it will work, and watch for any shader compile issues.

AVProVideo.zip

Ste-RH commented 1 year ago

Excellent. A few caveats.

1) You will have to replace that file if you update AVPro Video version again to a newer version. 2) I doubt OES mode will work.

RuudvanReenen commented 1 year ago

I've removed that reply, it seemed to work, but left eye kept switching between green and red when looking around, but not consistently. But for this app I'll just stay with 2.2.2, deadline is approaching. Majority of my work is with Unity 2021 and XR which works great, also with the latest updates.

Ste-RH commented 1 year ago

And you don't get that with 2.2.2?

You have the UpdateMultiPassStereo component on your camera, yeah? You are using MultiPass (not multiview), yeah?

Chris-RH commented 1 year ago

Closed due to lack of reply