WEKIT-ECS / MIRAGE-XR

MirageXR is a reference implementation of an XR training system. MirageXR enables experts and learners to share experience via XR and wearables using ghost tracks, realtime feedback, and anchored instruction.
Other
27 stars 4 forks source link

New UI scaler method needed for landscape mode (mobile) #1947

Open fwild opened 1 month ago

fwild commented 1 month ago

Since migrating to URP, the scaler method used does no longer work for landscape orientation, and we need to find a new way. This, at the moment, is fixed by not adding the camera order when starting in landscape mode, which leaves a non-transparent blue bar on top of the UI.

fwild commented 1 month ago

From the closed PR: https://github.com/WEKIT-ECS/MIRAGE-XR/pull/1944

Benedikt:

Unfortunately, this only works in portrait mode but not in landscape orientation. This is also were I am currently stuck.

When the device is in landscape orientation, the overlay gets stretched across the entire screen instead of positioning it on the right side. It also seems to mess up the input mapping as I can no longer click any of the buttons. Instead, their interaction area is somewhere else on the screen but not, where the visible button is. grafik

Apparently, there is a limitation in URP that overlays cannot be moved and scaled using the viewport rect, so our previous method of positioning the UI does not work anymore. https://forum.unity.com/threads/why-cant-overlay-cameras-use-viewport-rect.924932/

The suggested solution in the forum is to use a render texture but that seems suboptimal. Maybe camera projection matrices could help by directly manipulating the overlay camera image into position but that could also mess up the mapping of input coordinates when clicking on the buttons. Another solution might be to adjust the entire UI so that it always covers the entire screen. For landscape resolutions, we could then add a blank/transparent UI element on the left which pushes the actual UI content over to the right but that is quite a substantial change and could also interfere with inputs that should reach the objects in the scene.

So it seems like this line here does not have an effect anymore in the URP pipeline.

https://github.com/WEKIT-ECS/MIRAGE-XR/pull/1944/files#diff-72fd6680344304866a797542739b89d7fa2ec3974810953546c5a4eb718922e1R60