Closed tbirdso closed 2 years ago
Investigation results:
VolumeViewer
example works as expected in WebXR emulator browser extensionVolumeViewer
AR rendering works on mobile device, however the far clipping plane seems to be set very close to the camera. The volume is not displayed until the device is moved forward to be a few inches away from the virtual object.VolumeViewer
sample data so as to match the opacity/color functions: https://data.kitware.com/api/v1/file/624320e74acac99f42254a25/downloadVolumeViewer
. Unfortunately I won't have time to dig deeper than this for a few days.@floryst Reminder that this exists, no particular rush though. Would be helpful to have 1) any console printouts from volume rendering on Android and 2) your thoughts on parameters that might affect the clipping plane, which seems to be set to the default (far plane 1000.0)
I got the "Start AR" button to work with some CSS modification, and got a volume to show up in AR on the ar-apps
branch.
Touching the screen did throw the following error (index.js:437
):
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'buttons')
at index.js:437:62
at XRInputSourceArray.forEach (<anonymous>)
at Object.publicAPI.updateXRGamepads (index.js:422:28)
at _callee3$ (index.js:418:8)
at tryCatch (polyfill.js:5901:40)
at Generator.invoke [as _invoke] (polyfill.js:6135:22)
at Generator.prototype.<computed> [as next] (polyfill.js:5953:21)
at asyncGeneratorStep (Volume.js:11:1)
at _next (Volume.js:11:1)
at Volume.js:11:1
I'm also experiencing the clipping plane issue. The range should be much wider in order, otherwise it's difficult to use.
If I invoke pipeline.renderer.getActiveCamera().setClippingRange(10, 1200)
, the visualization is far more acceptable. The standard resetCamera()
works well for 2D views, but not so well with AR. One idea is to get the scene's bounding box, inflate it a bunch, and then pass it into resetCamera(bounds)
.
FYI @tbirdso to fix the clipping issue, we can add conditionals for the VR version of resetCamera and resetCameraClippingRange, as seen here: https://github.com/Kitware/VTK/blob/master/Rendering/VR/vtkVRRenderer.cxx
I experimented a bit with resetting clipping planes for AR last Friday. Increasing the far clip plane helped significantly and seemed to correspond neatly to real-world distances. However, the near clipping plane to anything farther than 0.1 or so seemed to reduce the far clip distance rather than moving the near clip plane outward, which was unexpected. Unclear if this is an issue specific to WebXR, need to experiment further.
Do we have an example for adjusting a camera's clipping planes without XR that I could use as a baseline?
There isn't any example that adjusts the clipping range IIRC. You can take the volume rendering example and adjust the camera clipping range manually via devtools, or add a small slider UI for near/far.
I'm not familiar with any behavior that would force the far clipping range to move closer if you reduce the near clipping plane to below 0.1. I would be interested to know if that occurs with the regular renderer.
I was not able to reproduce the clipping issue by playing around with the Cone example, the near and far clip planes were adjustable with setClippingRange
as expected. That is a bit of a relief.
I will investigate whether something strange might be happening with our physical-to-view transformation in XR rendering.
EDIT: I also get expected near clipping behavior when experimenting with data in VolumeViewer
without XR. I continue to get strange behavior in XR where I cannot adjust the near clipping plane to clip through volume data. Have not been able to pin down any underlying behavior that would lead to this.
@floryst I am getting a bit confused on this one. From what I can tell getClippingRange
is being called in VolumeMapper using parameters from openGLCamera. Those camera parameters do not seem to reflect the view-to-physical and physical-to-world adjustments made in Core/Camera. As far as I can tell OpenGLCamera
and Camera
are being treated as completely separate objects.
Could you please comment on how OpenGLCamera
is being used in VolumeMapper
? Am I missing something obvious as to how OpenGLCamera
should be composed with / inherit from Camera
?
model.openGLCamera.getRenderable()
should return a vtkCamera
. The OpenGLCamera represents the opengl scene-graph node for the camera, so getRenderable()
returns the vtk.js abstraction for the camera with which we interact.
Thanks, I think that makes sense. Maybe the problem does not lie there then. Will keep digging.
Collecting thoughts so far:
camera.setClippingRange(5 * physicalScale, 10 * physicalScale)
then the volume starts to clip out of the scene when the camera is 5m in physical space away from the volume.OpenGL/VolumeMapper
would explain why this issue only appears for volume rendering and not surface rendering. However, as far as I can tell the openGLCamera
object is getting the correct world-to-view matrix from the underlying vtkCamera
for rendering.~ EDIT: Seeing inconsistent behavior in AR
surface rendering example, clipping range seems to be entirely ignored.@floryst I am switching gears to focus on #2442. If you have the time I think I need more help to make progress on this issue, my knowledge of VTK's volume clipping procedures is limited and I'm hitting a wall on what I know to investigate. Would appreciate input from someone else who is more familiar with what could be causing the behavior I've described above.
:tada: This issue has been resolved in version 24.16.4 :tada:
The release is available on:
Your semantic-release bot :package::rocket:
Need
WebXR examples currently only encompass surface rendering. It would be useful to add an example demonstrating volume rendering as well.
Feature
Two changes are needed:
AR volume rendering was previously observed to work in the WebXR Emulator but the vtk.js scene did not appear when viewing on mobile.