Screen Space : Overlay render mode not available with Google Cardboard XR plugin

[subhajitRV]

Hi, We have recently migrated our Unity application from using the GVR SDK to the Google Cardboard XR plugin as an underlying VR framework for our VR application, to resolve the VR render spinning in iOS issue. We have noticed that there is a difference in how these VR SDKs work, The Unity UI render modes available for GVR were Screen Space -Overlay, Screen Space :Camera and World Space, allowing the UI to be overlaid and followed around with the user's view/camera, placed in the camera's view frustum and be placed in the World space around the user respectively. However, in the Google Cardboard XR SDK, only Screen Space: Camera and World Space rendering modes, for the Unity UI to be rendered, are available. We would like to know whether there is any workaround available to include Screen Space : Overlay mode with Google Cardboard XR, so that we can overlay our UI in the user's view and also be able to render & interact with it in VR, or whether this is an expected behavior of the SDK.

Thanks for the support.

[jballoffet]

Thanks for reaching out to us! As can be seen here, the Cardboard XR plugin just creates two textures and passes them to Unity in order to get the scene content rendered onto them. Bearing that in mind I'm not able to see a reason of why this is not working properly beforehand.

Could you please share a simple repro project with us where the issue is shown and the steps required to reproduce it? I guess a modified version of Hello Cardboard should suffice. Thanks!

[jballoffet]

Some more details in case they're useful for your use case. Currently, the Cardboard XR plugin performs texture overlays for the gear, the close button and the eye separator (vertical white line).

Here are some pointers to the code:

• Make Unity load the textures for us
• Pass into the display provider the textures above
• Here and here, those widgets are renderer as overlays of the VR view.
• Touch events are handled here.
• The rendering order (eye cameras and widgets) can be seen here.

I just want to point out that the Cardboard XR plugin currently doesn't provide an API at the moment to add overlay widgets but I believe that perhaps you can benefit from what we have and modify it to add more widgets as it can be easily extended by calling CardboardUnity_setWidgetCount(your_widget_count) and CardboardUnity_setWidgetParams() when initializing the XR provider.

HTH.

[Pithakote]

@jballoffet Thank you for the help! I was also looking into this with no avail, but I was able to make my own overlay widget and I hope you don't mind, I've posted what I've done to make it work for me in here. Hopefully someone will find this helpful!

I have been able to add my own overlay image as shown in the picture by the blue circle and I have been able to add my own functionality with it as well!

I did this by adding a new Rect in "Widget" class

In Widget class:

        public static RectInt VRButtonRect
        {
            get
            {
                RectInt rect = new RectInt();
                rect.x = ((int)Screen.safeArea.width) / 2;
                rect.width = DpToPixels(_buttonSizeDp);
                rect.y = (int)Screen.safeArea.yMax - DpToPixels(_buttonSizeDp);
                rect.height = DpToPixels(_buttonSizeDp);
                return rect;
            }
        }

        public static RectInt VRButtonRenderRect
        {
            get
            {
                RectInt rect = TranslateToSafeAreaFrame(VRButtonRect);
                int paddingPixels = DpToPixels(_buttonPaddingDp);
                rect.xMin += paddingPixels;
                rect.xMax -= paddingPixels;
                rect.yMin += paddingPixels;
                rect.yMax -= paddingPixels;
                return rect;
            }
        }

In the XRLoader class, added my own Widget and made sure to change the "CardboardUnity_setWidgetCount(4)" to 4 and when I set the widget parameters, I made sure to indicate which index should have which Widgets' parameters set:

        private static Texture2D _closeTexture;

        private static Texture2D _gearTexture;
        +private static Texture2D _cardboardModeTexture;
   [    

     //UNTOUCHED CODE SECTION
     ]
 internal static void RecalculateRectangles(Rect renderingArea)
        {
            // TODO(b/171702321): Remove this method once the safe area size could be properly
            // fetched by the XRLoader.
            CardboardUnity_setScreenParams(
                    (int)renderingArea.x, (int)renderingArea.y, (int)renderingArea.width,
                    (int)renderingArea.height);

            RectInt closeRect = Widget.CloseButtonRenderRect;
            RectInt gearRect = Widget.GearButtonRenderRect;
            RectInt mobileVRRect = Widget.VRButtonRenderRect;
            RectInt alignmentRect = Widget.AlignmentRect;
            CardboardUnity_setWidgetCount(4);
            CardboardUnity_setWidgetParams(
                    0, _closeTexture.GetNativeTexturePtr(), closeRect.x, closeRect.y,
                    closeRect.width, closeRect.height);
            CardboardUnity_setWidgetParams(
                    1, _gearTexture.GetNativeTexturePtr(), gearRect.x, gearRect.y, gearRect.width,
                    gearRect.height);
          +CardboardUnity_setWidgetParams(
          +       2, _cardboardModeTexture.GetNativeTexturePtr(), mobileVRRect.x, mobileVRRect.y, mobileVRRect.width,
          +         mobileVRRect.height); 
            CardboardUnity_setWidgetParams(
                    3, Texture2D.whiteTexture.GetNativeTexturePtr(), alignmentRect.x,
                    alignmentRect.y, alignmentRect.width, alignmentRect.height);
        }
     [    

     //UNTOUCHED CODE SECTION
     ]
            _closeTexture = Resources.Load<Texture2D>("Cardboard/quantum_ic_close_white_24");
            DontDestroyOnLoad(_closeTexture);
            _gearTexture = Resources.Load<Texture2D>("Cardboard/quantum_ic_settings_white_24");
            DontDestroyOnLoad(_gearTexture);
         +  _cardboardModeTexture = Resources.Load<Texture2D>("VRModeNot");
         +  DontDestroyOnLoad(_cardboardModeTexture);

In the Api class I made my own boolean to check if the custom button is being pressed or not:

public static bool IsMobileVRButtonPressed
        {
            get
            {
                if (!XRLoader._isStarted || Input.touchCount == 0)
                {
                    return false;
                }

                Touch touch = Input.GetTouch(0);
                Vector2Int touchPosition = Vector2Int.RoundToInt(touch.position);
                return touch.phase == TouchPhase.Began
                    && Widget.VRButtonRect.Contains(touchPosition);
            }
        }

I then called those booleans from my custom class's Update function which checks for those booleans from the Api class :

#if !UNITY_EDITOR

                if (Api.IsCloseButtonPressed)
                {
                    ApplicationQuit();
                }

                if (Api.IsGearButtonPressed)
                {

                    Api.ScanDeviceParams();
                }
                if(Api.IsMobileVRButtonPressed)
                {
                    CustomFunction();
                }
                if (Api.HasNewDeviceParams())
                {
                    Api.ReloadDeviceParams();
                }
#endif
    }

[jballoffet]

Thanks for sharing, @Pithakote !

[CYCPeter]

hi, follow @Pithakote method, I can add new button. Is it possible if I want to hide/show the new button through parameters (like scene name). how can I do? thanks

[johnfriedrich]

Came across this aswell. We would like to add a button that can be hidden once pressed. Did not find a way to hide it. Also are there other ways to add a button? Using a normal Canvas with different render modes did not work