search

immersive-web / proposals

Initial proposals for future Immersive Web work (see README)
95 stars 11 forks source link

Do we need a simple on/off occlusion solution for a credible "AR min-bar"? #45

Open thetuvix opened 5 years ago

thetuvix commented 5 years ago

We discussed at the January F2F that the hit-testing sample in immersive-web/webxr#493 always prefers hitting the virtual object vs. world geometry due to not having a WebXR occlusion solution, meaning virtual content will always end up on top of walls anyway. However, it doesn't sound like AR apps will produce intuitive experiences for users if things always end up available through walls.

Does this suggest that perhaps a simple on/off occlusion solution is needed to meet the credible "AR min-bar" for WebXR?

Originally posted by @thetuvix in https://github.com/immersive-web/webxr/pull/493

toji commented 5 years ago

Neither of the prominent mobile AR platforms (ARCore/Kit) have any form of occlusion at this point, which I think pretty firmly indicates that it's not a min-bar requirement.

thetuvix commented 5 years ago

To what degree do ARCore/ARKit apps receive wall planes during practical user operation on those platforms? An app receiving wall planes on those platforms could do their own occlusion - however, we've scoped out giving apps that geometry data for 1.0. Could a more scoped version of that just enable occlusion?

toji commented 5 years ago

The planes exposed by those platforms are geared towards object placement and not suitable for occlusion. I guess you could technically use them as occluders but the effect would not be very stable or well fit and it would probably look worse than simply not occluding at all.

blairmacintyre commented 5 years ago

I think that sample code is broken. If we are going to provide an example of how to combine virtual and real hit test results, the sample code should do both hit tests, and then return the closer one.

Visual occlusion is orthogonal to this issue.

For that, I am ambivalent; current versions of ARCore/ARKit return planes that are inadequate for occlusion, but it's easy to make the argument that while inadequate, they are better than nothing.

I suppose a bigger question is "how would this be used" once we are (potentially) exposing this information to developers.

Assuming a developer has access to this geometry, it is very likely they will use it for occlusion: it's unlike a generic app will inspect the geometry to see if it's a dense mesh (like HMDs are returning) or a sparse set of planes.

If the UA does not provide the mesh to the app, where is this "simple on/off occlusion" being done?

bricetebbs commented 5 years ago

I agree that we could use better framing around how we expect people to use HitTest. I filed #hitest/34 about this a while ago.

The point above about the occluders not being stable is also a good one. In the experiments I've done, the initial estimates of the plane geometry are pretty lousy or bloated but they can improve over time. We don't really have any way to surface this yet.