Right-left handed coordinates problem

[AqlaSolutions]

After I updated SharpNav in our project all the bots are trying now to go outside of the physics map. I presume that X coordinate is now inverted. But previously the same code worked fine! So did you introduce some changes on this?

Can you add please some setting for deserializer or somewhere else so we can have the same X axis as before?

[AqlaSolutions]

Hmm.. or may be it's because we previously generated navmesh right in the app and the model was loaded with our "correct-handed" axis. But now the model is loaded in the "Examples" project, serialized there and deserialized on our side.

[Robmaister]

SharpNav assumes left-handed (Y up). Nothing should have changed on our end. It's almost definitely because you serialized in one coordinate system (Examples) and deserialized in the other (your program). For the sake of simplicity, I'm going to say that you should assume that coordinate space for all SharpNav files.

Also the generation process is dependent on that same coordinate system, so I'm really not sure how you'd be generating a mesh in a right handed space and be getting any decent result out...

[AqlaSolutions]

So are you going to provide any setting to toggle this or should I make a wrapper around NavMeshQuery on our side?

Y-up is everywhere but left/right-handed is different between DirectX and OpenGL.

[AqlaSolutions]

Sorry for being a bit misleading, looks like I have to tell you the full story)

1. Initially the model is saved in 3DsMax.
2. SharpNav library just treats the coordinates as they are, so no difference for left/right handed.
3. In your Examples app those coordinates are used to render the model and navmesh (everything is fine here).
4. On our side we need to deal with another-handed axis because Unity3D uses it. So we load the model with inverted X axis and the order of vertices.
5. Previously we used that model to generate NavMesh that we need but now we generate everything in Examples app and the model is not inverted there.
6. If we change the loader on Examples app side it will generate correct (for our app) navmesh but the renderer will show the model mirrored.

So we need to generate NavMesh in one-handed system, then open it in our app where we need it another-handed.

Can you help us to decide how we can deal with this situation with SharpNav beside maintaining a wrapper of NavMeshQuery on our side (and I'm not sure whether it will work)?

[Robmaister]

Looks like I got my terminology mixed up for a second there. Unity and SharpNav both use a right handed coordinate system (Y up/X right/Z forward) as opposed to the left handed coordinate system (Y up/X right/Z backwards) and there are Z up variants of both.

Regardless, the reason why your X axis is negative is because you're generating the navmesh before flipping the X axis in Unity, while every other system in Unity works on the model after you flip the X axis (including genrating the navmesh as you used to do). If you flip the X axis in 3ds Max instead of Unity, this problem will go away.

So the solution is to fix whatever underlying reason you have for flipping the model on the X axis in Unity.

If you don't want to do that, consider modifying Examples (or writing your own importer/exporter based off Examples) that transforms all of your input geometry to flip the X axis through an IEnumerable<Triangle3> with some of these helper functions: https://github.com/Robmaister/SharpNav/blob/master/Source/SharpNav/Geometry/TriangleEnumerable.cs