CutPlane and CutPlaneFilterMask RayCast support

[WikkidEdd]

Hi

We're finally getting around to implementing raycast collision against remotely rendered models (as opposed to bounding box collision). We've used the showcase app as a reference and got the basic raycasting working very quickly so that is great. However, our app already uses both CutPlane and CutPlaneFilterMask which is not currently supported for raycasts.

Your documentation suggests that we adjust the starting point of the ray to be on the cutting plane and cast from there, but that won't work when using the CutPlaneFilterMask. The only way I can see of supporting this with the features available is to use the HitCollectionPolicy.ClosestHits (aka "All") and discard the hits which don't meet the CutPlane and CutPlaneFiltermask criteria.

Is this a realistic solution given that the documentation says that this is the least efficient raycast mode? Not to mention the edge cases we'll need to handle due to the fact we can only get 256 hits per raycast.

Obviously, in an ideal world, this would all be taken care of server side, but failing that perhaps an additional raycast feature could make it more efficient.

Raycast against layer - We have a similar problem for raycasting local models that use cutting planes. In this scenario we solve it by put all the objects effected by the cutting plane on a separate physics layer. This lets us do 2 efficient raycasts which only return the closest hit. One using the full ray length against everything that doesn't have a cutting plane and one with the starting point on the cutting plane which only hits objects effected by the cutting plane. Then we can just take the closest of the two. If you could allow something like this based on the CutPlaneFilterMask we could achieve something similar for remote models too.

Edit: Oh, I just spotted this is part of the raycast struct. That sounds like exactly what I want. Will it be active anytime soon?

        /// <summary>
        /// Reserved for future use.
        /// </summary>
        public uint CollisionMask;

It would be great to get your thoughts.

Thanks,

Edd

[FlorianBorn71]

Hi Edd, we do in fact have a backlog item for raycasts to respect the cutplanes, but to be honest, it has a low priority at the moment, because there are ways around that with a bit more client-side code. Specifically, the method you mentioned wrt returning 'All' hits. Don't over-index on the perf implication, it should be fine as the major difference to 'Closest' is only that we have to send all the results. So this should be the way to go for you.

The CollisionMask property is currently unused. The problem here is that we don't see a good way to author the value on the counterpart side. Would it be a per-part property in the fbx file? Or can be set as a hierarchical state override?

If you think the local evaluation is not sufficient, you can create and upvote an item here: https://feedback.azure.com/d365community/forum/46aa4cc0-fd24-ec11-b6e6-000d3a4f07b8

Cheers, Florian

[WikkidEdd]

Thanks for the quick response.

Can you clarify the maximum number of hits returned via ClosestHits? The documentation has two values; the MaxHits comment says 256 whereas the constant MaxNumRayCastHits says 1024.

Given that the overrides are hierarchical, in order for us to determine whether the CutPlaneFilterMask is applied it would require us to walk up the hierarchy and inspect all the HierarchicalStateOverrideComponent components to determine is they are masked or not. Would you expect there to be any performance to be ok running that up to 256/1024 times per raycast or would you suggest we create a cache for EntityId -> CutPlaneFilterMask look up?

I think setting the CollisionMask via Hierarchical State Override makes the most sense to me. It's no different from disabling collision via that method IMO. If you could enable this I think it would take a lot of the pain out of the above method.

[FlorianBorn71]

Thanks for the hint with the max count inconsistency in the docs. 1024 is the correct value, I will make sure the documentation is fixed. I would first try a more or less naive version where you traverse up the graph until you find a state override component. It could be fast enough because I don't think you'll have that many hits in one raycast. With the cache you might run into problem with updating it, e.g. upon unloading the model. If the whole approach turns out to be too slow, please let us know, we can see if we can somehow prioritize the server side solution.

[WikkidEdd]

I would generally agree with you regarding not having that many hits from one raycast, but when we're talking about a serrvice that can support 500m+ tris it becomes much harder to put a solid estimate on the expected number of hits. :)

I agree with regards to the cache, definitely becomes a pain to maintain so happy to start with the simplest approach.

Can you confirm if the raycast will return hits against backface triangles?

[FlorianBorn71]

Can you confirm if the raycast will return hits against backface triangles?

Yes, raycasts use the same backface clipping as for rendering

[WikkidEdd]

This is all working as expected so far and no major performance issues as yet. I'm capturing the max number of raycast hits for a single ray and with an engineering model of about 8m tris I'm getting up to about 50 hits max, so it's ok, but makes me a little uneasy that it's not a fixed client size cost.

The normals that you get for the backface collisions appear like are flipped to be facing you. Is that correct?

Is there any way to get it to give me the actual normal (i.e it's facing away from me and I want the normal of it facing way too) or some flag to tell us if it's a backface?

Knowing whether it's a backface is useful. For local models which have cutting planes, if it hits a backface (assuming manifold geometry) we move the hit point to the cutting plane, this gives the feeling of the objects being solid and makes the UX a bit nicer. We can't do that for remote models unless we know if we've hit a backface.

[FlorianBorn71]

50 is already more than I would have expected ;-) Note that this number does not scale with the triangle complexity (tessellation level) of the model, but only with its depth complexity. Unfortunately, the normals need to point out, because the lighting would look very unnatural when you have a model with combined back- and front faces. So currently there is no way to find out whether a hit was a back face in the source model.

Would the lighting be a concern for you, just in case there was an option at export time to flip normals of back faces? Or would turning off all back faces, both for rendering and raycasting, be an option to begin with?

[WikkidEdd]

Admitedly, the 50 was me trying to put the ray through what looks like the most complex bit of the model to work out a worse case. :) Yes, I appreciate that it's the number depth layers which impact the raycast not the tri count as such, but there is likely to be some correlation. It would be reasonable to assume that a 500m tri model would have a higher max raycast hits than an 8m tri model.

Perhaps I need to clarify what I'm asking for with regards to backface normal:

Here we have a ray shown in red travelling from left to right. It hits a sphere, Hit 1 returns what is an unambiguous normal of the hit point. Hit 2 is perhaps a bit more ambiguous behaviour. In Unity when you turn on Physics.queriesHitBackfaces it will return the normal in blue, this is the normal of the mesh at the hit point. Whereas ARR seems to return the normal in green which is the normal as if the geometry is actually duplicated to have a triangle facing inwards.

My desired output in the RayCastHit would be the normal in blue regardless of how the model is rendered or lit.

[FlorianBorn71]

Got it. Just fyi, we are discussing options internally. Flipping the normal would be a breaking change, but there are other ideas as well.

[WikkidEdd]

I should add that this isn't a blocker but more of a nice to have.

[FlorianBorn71]

Hi Edd, I'm happy to announce that this feature made it into the new client release 1.1.14 https://github.com/Azure/azure-remote-rendering/releases/tag/1.1.14

From the changelog: The RayCastHit struct now has a new member HitType which defines what was hit by a ray cast: TriangleFrontFace, TriangleBackFace or Point.

Cheers, Florian

[WikkidEdd]

Great, we'll check it out soon!