Avoid needing a special copy of the navmesh

[justonia]

Having prototyped a similar teleport mechanic using Unity's NavMesh API, I had success not requiring a separate bake-and-save of the mesh geometry. All I had to do to make the nav mesh work for both AI and the player was to:

• Properly set up and query area masks. E.g. add an area mask called WalkablePlayer and flag desired geometry with it. AI agents would then have their agent mask set to both Walkable and WalkablePlayer while the manual queries to determine player destinations would only use WalkablePlayer
• Put a restriction of the surface normal of a valid teleport destination to avoid being on a slanted area (e.g. Dot(Vector3.up, hit.normal) ~= 1f).

I'd recommend trying this out instead of re-implementing a nav mesh yourself.

[Flafla2]

An interesting approach - definitely better / more optimized than what we have now. Right now I'm deciding between two directions for this project:

Option 1: Your approach

Pros:

• No reimplementation of Navmesh API
• Already optimized from Unity's work
• No need to change settings / rebake to get the Navmesh ready
• More control over what areas are walkable / not walkable

Cons:

• Restricted to prebaked Navmeshes (ie no procedural levels)
• Restricted to Mesh Renderers / not using simpler colliders (an often requested feature)
• Agents needs to be reconfigured (to account for Walkable and WalkablePlayer)

Option 2: Use Recast+Detour for Navigation (See #9)

Pros:

• No reimplementation of Navmesh API (Detour solves this problem)
• Already optimized API (and open source)
• As a stepping stone, we could integrate Detour first (using Unity's navmesh) then integrate Recast for baking (allowing us to skip the rebake step)
• Once recast is implemented procedural levels would be possible (ie baking at runtime)
• Once recast is implemented we could calculate the Navmesh using Colliders instead of meshrenderers
• No need to reconfigure Navmesh agents

Cons:

• I would have to port Recast / Detour (not too hard, I would just compile it and use C#'s [DllImport()] and marshalling system)

Both approaches are better than what we have now. However to me it seems like option 2 is better in the long term. What are your thoughts on this?

[justonia]

Well, option 1 can be accomplished in hours and has no external dependencies. Option 2 is enormously involved and, as described, requires a native plugin compiled per-platform. For option 2 you are much better off using a C# implementation of Recast, such as: https://github.com/Robmaister/SharpNav

That said, if you want the best of all worlds you should consider how a valid point gets picked to be implemented via a strategy pattern so that different implementations may be used. If done that way, it'd be trivial to support the existing implementation, option 1, third party assets like Apex Path, or whatever other method one could dream up.

[Flafla2]

I didn't know about SharpNav, thanks for the link!

I think in the short term I'll implement option 1 - as you say it is trivial to implement and solves the optimization problem for now. Looking forward though a thirdparty system is more than likely the way to go.

[Flafla2]

Ok, so I spent some time today implementing your idea (option 1 in my post above). It works, and I'm definitely seeing performance improvements as expected. However I don't think the idea solves the problem of decoupling the AI Navmesh totally from the Vive Navmesh. In short, your method produces some minor errors in the Navmesh around sloped areas. The cause of the errors is based in the fact that it is impossible to set "Step Height" to zero and "Max Slope" to something greater than zero in the navigation settings. In general this means that you can get Navmesh bakes that look like this:

As you can see the nonzero step height moves one of the Navmesh vertices onto the slope. As you say, we can compare the dot product of the normals of each triangle with the up vector to at least remove the slopes, but we are still left with an inferior mesh (visually). Below I compare the navmesh using your technique to one with Max Slope = 0 (what we have now):

The cause of this problem is perhaps more apparent when viewing the Navmesh from above:

The question is whether or not these artifacts are acceptable. Honestly, I don't believe they are, so in the end I'm probably going to end up integrating a third-party navigation system. However, the implementation of your solution is much more performant, and also allows users to cut out large parts of complex scenes using the layer mechanism regardless of the aforementioned slope problem. With this in mind I'm still going to work towards stabilizing / integrating the revision for now.

[justonia]

Ah yes I see the problem, it's definitely noticeable when it comes to visualizing the mesh. How does it feel in this example when doing the destination picking (ignoring the visuals of the mesh itself and using the dot product sanity check)?

To play devil's advocate when it comes to the visuals, if I were shipping a game using this or a similar solution I would be quite fine baking and then use a tool like ProBuilder to clean up the generated mesh for the nav visual. Visually speaking, even the correct version isn't always going to be aesthetically pleasing without touch up.

And thanks for prototyping this, I certainly had no expectation you would consider testing an implementation five hours after opening the issue!

[Flafla2]

Hi, a quick update on this issue.

I have decided to move forward with your "option 1" and have implemented it into version 2.0.0. You can get the new version here.

2.0.0 supports navmesh areas and automatically culls navmesh faces that don't face directly upwards. For raycasting, it uses Unity's Physics.Raycast as there is no such thing as a true NavMesh Raycast (Unity's NavMesh.Raycast doesn't actually perform a 3D raycast, it projects the two points on the mesh and attempts to draw a line through them).

I'm closing this issue for now as the most pressing part of it (optimization concerns) have been addressed. Please reopen if you feel it is necessary!