Add Passthrough to Pico family of headsets

[RedXi]

I am using Open Brush with my Pico 4 VR headset and have noticed that the AR mode is not functioning correctly. When attempting to enable the Passthrough mode, I only see a black screen instead of the environment through the VR headset's camera.

Here are additional details that may be relevant:

• VR headset model: Pico 4
• VR headset firmware version: 5.5.0
• Open Brush version: 2.2.0 (I have also tested older versions where the issue persists)

Steps to reproduce:

• Start Open Brush and connect the Pico 4 VR headset.
• Go to Open Brush settings and enable the AR mode with Passthrough mode.
• Instead of seeing the environment through the VR headset's camera, I encounter a black screen.

I have tested other applications and confirmed that the camera on my Pico 4 VR headset works properly. Hence, it seems to be a specific issue with Open Brush's AR mode.

I would greatly appreciate any assistance you can provide in resolving this problem. If additional information is required, please let me know.

Thank you!

[mikeskydev]

Hi,

Proper support hasn't been added for Pico passthrough yet due to a blending issue on some of our more fancier brushes. Some people have developed a fix by reducing the passthrough opacity but I want to fix it properly, otherwise we end up with a bunch of issues saying 'passthrough is dim on Pico 4' instead! Hope to have this solved soonish.

[RedXi]

Hi mikeskydev,

Thank you for your prompt response. I appreciate the information you provided regarding the current lack of support for the Passthrough mode on the Pico 4 headset due to technical issues.

I completely understand the importance of developing software of high quality.

I genuinely hope that the problem can be resolved soon, as I am eagerly looking forward to the release of Open Brush in Passthrough Mode for the Pico 4. Given that the Pico 4 headset offers a wireless experience, "Open Brush" would indeed be a great fit for it.

Best regards, RedXi

[PGSCOM]

Same

[huihuangex]

I hope it can be resolved as soon as possible

[nickbisesi3]

Any updates on this? Would love to use passthrough on my pico 4

[mikeskydev]

We recently worked out the technique that allows our quest Passthrough to function correctly, so we're hoping we can duplicate it for other platforms really soon!

[MatteoFith]

Hi @mikeskydev, I am working on a proof of concept for Pico 4 with passthrough capability. I'm encountering the blending issue that prevents brushes from being displayed.

Do you know of any brushes that currently can be used with Pico passthrough? I'm not looking for a fancy brush; a very simple one would be perfect for my needs.

[mikeskydev]

Hi, that's great! OilPaint should work just fine. I've been poking some more at this but there's a lot to do, so if you're going to look into it further I'd love to chat more. Are you in our Discord?

[MatteoFith]

Yes, I have just sent you a DM.

[mikeskydev]

I think we're finally going to get a "proper" solution!

The missing piece was being able to control the blending of the default projection layer, which will be provided with Unity's imminent release of the composition layers package: https://discussions.unity.com/t/openxr-control-over-eye-camera-layer-alpha-blend-like-ovrplugin/1510768

Pico uses XR_FB_passthrough underneath (because it's a decent solution, fair play for not reinventing the wheel), and I've already engineered a way to use it with composition layers.

[mikeskydev]

Got tired of waiting for Unity and did some hacking in a test project :D Here's some Open Brush shaders running in passthrough on Pico, blending correctly. Going on pat leave but will get to work on this properly once I'm back.