XR_KHR_composition_layer_equirect2 ambiguities

[amalon]

The mapping of the swapchain subimage to the inside surface of the partial sphere by XR_KHR_composition_layer_equirect2 is ambiguous. It could benefit from a diagram as provided for XR_KHR_composition_layer_cylinder.

For example, some questions:

• what direction is the "0 radian angle" of the partial sphere when angles are less than their maximum extents?
  • Towards negative Z of the pose?
• what should the extremities of provided angles (corners of partial sphere) map to?
  • The corners of the swapchain subimage (so the subimage fills entire partial sphere)?
  • The mapped coordinates assuming a full sphere, i.e. based on the vector from the pose (such that the partial sphere shows only a window into the equirectangular subimage)?
• what direction should the center U coordinate of the swapchain subimage be mapped?
  • Towards the "0 radian angle"?
  • Fixed relative to the space?

Monado currently doesn't scale, position or orient the partial sphere or the texture mapping at all (and the points on the partial sphere map directly to the subimage based on the direction rather than the angle within the partial sphere), so I'm left guessing what the correct behaviour was intended to be.

[rpavlik-bot]

An issue (number 1793) has been filed to correspond to this issue in the internal Khronos GitLab (Khronos members only: KHR:openxr/openxr#1793 ), to facilitate working group processes.

This GitHub issue will continue to be the main site of discussion.

[casseveritt]

The equirect sphere is at the origin of its local space with vertical angles of pi/2 and -pi/2 on the y axis. At the vertical angle 0, the zero horizontal angle is on the -z axis, just like for the cylinder layer.

[casseveritt]

The above is in the space of the sphere. The pose member of this layer is used to position and orient the sphere relative to some other space.

[amalon]

thanks @casseveritt so to confirm, does it map the whole subimage to the visible portion of the sphere (like the cylinder layer), or is the texture mapping purely based on the angles. I would have thought the former would make sense since the latter could still be done by changing the subimage rectangle.

Its worth noting too that the original equirect extension is similarly ambiguous. I presume the scale applies before the bias, and that resulting texture coordinates outside of the subimage are clipped. Given the two extensions are supposed to be roughly functionally equivalent I presume a mapping between them would be something like this?

equirect.scale.x = equirect2.centralHorizontalAngle / (2*PI)
equirect.scale.y = (equirect2.upperVerticalAngle - equirect2.lowerVerticalAngle) / PI
equirect.bias.x = 0.5 - (equirect2.centralHorizontalAngle/2) / (2*PI)
equirect.bias.y = 0.5 - equirect2.upperVerticalAngle / PI

or maybe for OpenGL this?

equirect.bias.y = 0.5 + equirect2.lowerVerticalAngle / PI

As far as I'm concerned its also ambiguous how the V texture coordinate is interpreted. The cylinder layer extension puts V=0 at the top of the cylinder (like Vulkan would interpret), but the OpenGL extension says the bottom-left of swapchain image (not necessarily physical objects in VR space) should be interpreted as the coordinate origin unless otherwise stated by an extension. Does that mean an OpenGL app would have to render to that swapchain upside down for the runtime to show it the right way up? I presume equirect should match whatever cylinder does.

Sorry to be so pedantic!

[amalon]

Hmm, sorry, I think I've got that mapping backwards, and scale should be the inverse. Maybe somebody who knows can specify the mapping between equirect and equirect2 and how the bias and scale are applied to what ranges of 2d coordinates.

[casseveritt]

Mapping eqr2 back to eqr parameterization would be:

scale = { (2*PI) / centralHorizontalAngle, PI / (upperVerticalAngle - lowerVerticalAngle) };
bias = { (1 - scale.x) * 0.5, (upperVerticalAngle / PI - 0.5) * scale.y};  

The OpenGL ES layout of these textures does introduce a "y flip" issue if you're rendering to them. This is one of those cases where the original content often came from image sources (e.g. video decoders) where the origin was already upper left. To accommodate different source conventions, we added FB_composition_layer_image_layout.

[casseveritt]

Also to your question:

does it map the whole subimage to the visible portion of the sphere (like the cylinder layer), or is the texture mapping purely based on the angles

The angle parameters define where the layer exists. Regions outside those angles should not map to anything visible. The subimage only defines which portion of a swapchain maps to that region. The image rect corners correspond to the angle corners in the sphere's parameter space.

[amalon]

Thanks @casseveritt, I think that clears up most of it. One quick clarification regarding the old equirect extension: are scaled & biased coordinates outside the range 0.0-1.0 treated as invisible too?

[casseveritt]

Scaled and biased coordinates outside the imageRect are treated as invisible in the original equirect extension. If your image rect was the whole image, then yes, outside that range would be invisible. But if you were only using part of the texture image, outside of that rect would be invisible.