Create a specification for rigid body physics

[jratcliff63367]

A number of people and companies have expressed an interest in having a GLTF extension which can describe rigid body physics.

A number of popular physics engines are widely used, each with a differing API and data serialization format. While many concepts between them are the same, there also tends to be significant differences between them as well.

Past efforts to create a standardized physics data definition have ended up with something so watered down that it was of questionable value, or so highly specific to one physics engine, that it could not be easily used by others.

I see the goal of this effort to create a data spec which represents a 'union' of the features of a number of popular physics engines; such that those things which are in common between them use a shared DOM (data object model) and those properties which are unique to each can still be specified and, conceivably, ignored by other physics engines which do not need them.

Even just the concept of a 'physics engine' can be challenging, as the definition can be wide ranging in scope. Some physics engines support soft bodies, particle systems, fluid dynamics, complex machines, and more.

While it would be desirable to support all features for all physics engines, that is probably unrealistic as an initial goal.

I would like to set the phase one goal to simply defining the basic components shared by any rigid body dynamics system; which is largely collision geometries, physical material properties, simulation settings, rigid body settings, collision filtering, and joints.

We should probably identify features that go beyond this, but leave them for future extensions efforts. Future work should also address the specific needs of robots which include cameras, sensors, grippers, transmissions, and a lot complex machine data.

Typically a DOM for a physics engine is similar to the scene graph you would see for rendering data; there are meshes and geometries which can be instanced at different locations and scale and they can be linked together with joints. There may need to be a way to specify a named binding between a physics representation and the visual representation it is associated with. Likely a parallel scene graph which simply describes those bindings will be needed.

The basic components of any rigid body system can be described in the following outline:

• Simulation properties
  • Each physics engine defines certain global properties which control how that system performs. Many of these properties tend to be unique for each different physics engine supported. Includes things like gravity, solver iteration count, and other global settings.
• Data Resources (objects with large amounts of binary data)
  • Triangle meshes
  • Convex Hulls
  • Heightfields
• Physics Materials
  • Defines physical properties of a material surface, friction, etc.
• Geometries (physical geometric representations for rigid bodies or parts of a compound rigid body)
  • Plane
  • Box
  • Sphere
  • Capsule
  • Cylinder
  • Triangle Mesh
  • Convex Hull
  • Heightfield
• Rigid Body (can be static, dynamic, or kinematic)
  • Each rigid body has one, or more, geometries and their associated unique properties; such as what material to use, what scale, etc.
  • A large array of physical properties are associated with each rigid body (mass, inertia tensor etc.)
• Aggregates
  • A collection of rigid bodies which can be treated as a single group.
• Joints (a joint is a type of constraint which joins two rigid bodies together)
  • Fixed
  • Spherical
  • Revolute
  • Prismatic
  • Distance
  • D6
• Body Pair Filters
  • Defines a sequence of object pairs which should or should not interact in the physics simulation.
• Collision filter logic
  • Describes the code/logic for how collision filtering between two rigid bodies should be interpreted.

The initial phase of this project will simply be requirements gathering. We have some graduated students who will be evaluating a number of well known physics engines to understand the data object model requirements for each. Once those are gathered we can try to create a 'union' of these specifications as well as identify engine specific properties. The last phase will be to actually create the schema itself and submit it for review. This process could take weeks to months to complete, and is only starting today.

Physics engines under consideration are:

• PhysX SDK : from NVIDIA
• FLEX SDK : from NVIDIA
• Bullet SDK : (open source by Erwin Coumans)
• Newton physics engine : (open source by Julio Jerez)
• Havok : from Microsoft
• Mujoco : Commercial physics engine, popular within the robotics community

[Moguri]

A rigid body physics extension has already been started as #1104. Maybe we can build from there?

[pjcozzi]

@jratcliff63367 thanks for kicking off this effort, it is much appreciated! Adding @akhabbaz and @rms13 to the conversation.

[pjcozzi]

@Moguri it would be great to see if there are collaboration opportunities and perhaps end up with one extension.

[jratcliff63367]

Yeah, I'm fine with merging the efforts. That one seemed specific to Blender only? I think the larger goal is to be more general to a wide range of physics engines.

[Kupoman]

@jratcliff63367 That extension is not meant to be Blender specific, and tries to generalize physics settings when possible. The reason it has the BLENDER prefix is because blendergltf is currently the only exporter using it. When multiple exporters and importers start supporting it, it can be promoted to an EXT or KHR extension.

[jratcliff63367]

Ok, this seems great then. We are looking for engineering resources to continue this effort so that it can encompass a number of popular physics engines as well. I'm trying to rustle up some college interns now.

[jratcliff63367]

Here's an example of a skeletal mesh and rigid body system I would like to be able to fully describe in GLTF.

https://www.youtube.com/watch?v=hw1TpxfglrE&feature=youtu.be

[donmccurdy]

I see the goal of this effort to create a data spec which represents a 'union' of the features of a number of popular physics engines; such that those things which are in common between them use a shared DOM (data object model) and those properties which are unique to each can still be specified and, conceivably, ignored by other physics engines which do not need them.

While it would be desirable to support all features for all physics engines, that is probably unrealistic as an initial goal.

I am a bit nervous about this broad extension scope. glTF has so far had a lot of success with focusing on common features that can be implemented consistently across many engines. Is there a (still useful) approach to physics that would allow us to do the same?

If a particular physics engine wants to jump in and propose their own extension (building on this one), e.g. BULLET_physics_extras, they can certainly do that. But if everything is defined in a single superset physics extension, it becomes very hard to ensure consistent implementations, as glTF's extensionsRequired and extensionsOptional are no longer useful indicators of support.

[jratcliff63367]

I was envisioning something where you specify the common properties and anything engine specific can be added on as additional key/value pair tags. I'm not really familiar with JSON schema specs yet, but that's fairly common in XML specs.

[meshula]

Sometimes the motivation to incorporate physics in a graphics specification comes from an observation that it's nice to represent arbitrary polytopes with vertices edges and faces, and hey, graphics representations do that too, but the end goals are very different, and I think that ultimately the equivalence is superficial. I say this because I did a couple of physics engines at LucasArts where initially I went in with the goal of sharing graphics data with the physics engine, but it never turned out that the data could be shared in a practical sense. Physics representations by necessity are simpler and tend to bound an object rather loosely, and hugely benefit from implicit formulations. I know I'm belaboring the obvious, but I wanted to be clear about where my objection is coming from.

That said, I'm not meaning to rain on the parade. To the contrary, I am enthusiastically in support of the creation of a physics schema that can be independent of a particular engine. I've always been hopeful and then disappointed when things like PAL appear and then don't really gain traction. I just don't feel that gltf shouldn't take on more than representations of graphics state and things to render. I'm wondering if it might be a good idea to form a Physics Interchange oriented group to come up with a specification, independent of gltf, for broader use.

Long story short, all the goals you've laid out are awesome, I think they deserve their own, independent of gltf, effort.

[vpenades]

Please, take BepuPhysics for consideration to the list of physics engines. It's one of the few 100% C# free physics engines around

[Moguri]

@jratcliff63367 What's the current state of this proposal? I would like to move #1104 along so we at least have rigid body physics, but I would still prefer to collaborate on physics in general.

Maybe it makes sense to break this general physics extension into multiple physics extensions (e.g., rigid bodies, soft bodies, constraints)? This may make it easier on consumers if physics isn't an all or nothing deal (e.g., start with rigid bodies, then start implementing something like constraints). A consumer could ignore most of the extension, but it is odd to list support for an extension if support for large chunks are missing.

[jratcliff63367]

Here is the current status. I'm sorry it isn't further along, especially with the JSON stuff, but I'm also using this as part of a work project and that project has some different constraints and priorities. In particular, for the work stuff I have been told I need to support google protobufs, so there's been more progress on that than with the JSON schema at the moment.

That said, progress on the overall data-object-model has moved pretty far. I have a data definition for most of basic rigid body types. The missing parts are that I have not yet fleshed out all of the properties for the various joint types. That is still a todo item. The data definition includes a full hierarchical scene graph, easy to parse and understand.

Regarding properties specific to individual physics engines, I have added a generic 'property bag' called 'AdditionalProperties' where you can add key/value pairs for individual physics engines.

For example, say 'Bullet' has a bunch of properties specific to it when defining a scene. You can just add those properties to the 'AdditionalProperties' array in the Scene class. Same thing for 'PhysX' or any other physics engine.

Here is a link to the current data-object-model in a Google spreadsheet. The major missing component is detailing all of the properties for each individual joint type.

https://docs.google.com/spreadsheets/d/118I5kdu2XT-6wfCG044937xfEKDyX2oNg04G8Wqi6o0/edit?usp=sharing

I am not addressing soft-bodies with this spec. My current goal is to limit this specifically to rigid bodies.

I have a tool called 'CreateDOM' which parses this google spreadsheet and, the goal, is that it will automatically generate all of the C++ binding code and serialization schemas necessary to operate on the data.

For example, here is the auto-generated C++ header file for the spreadsheet linked above:

https://github.com/jratcliff63367/CreateDOM/blob/master/PhysicsDOM.h

It is important to note that this C++ header file is in POD format (plain-old-data). It has no depedencies on the STL; it's just the raw data and is safe to pass across DLL boundaries.

The implementation class (which is still a work in progress), is used to build, copy, move, iterate, manipulate, insert, delete, and otherwise modify the data.

Here is a link to what the implementation/reflection class looks like. This is not yet complete, and is still undergoing a lot of improvements. This code, too, is 100% auto-generated from the Google spreadsheet specification.

https://github.com/jratcliff63367/CreateDOM/blob/master/PhysicsDOMImpl.h

I realize that the focus for GLTF is JSON but, as I said, my internal priority is to get Google protobufs support.

Here is the auto-generated protobufs schema for the Google spreadsheet above. It is the design goal that this tool will, likewise, auto-generate the GLTF JSON schema as well.

https://github.com/jratcliff63367/CreateDOM/blob/master/PhysicsDOM.proto

My next tasks are:

• Finish adding all of the properties for the various joint types
• Finish auto-generating the implementation class; which lets you construct, move, copy, insert, delete, and iterate on the contents of a DOM representation.
• Auto-generate the GLTF compatible JSON schema
• Auto-generate the JSON serialization and deserialization code
• Finish implementing the demo app which can load any fully specified DOM and simulate it in a physics engine (some progress has already been made on this task)

I have some other requirements as well for protobufs and XML support, which are not important to this group but necessary for my internal project. I also intend to support converters for popular robotics file formats such as URDF, SDF, and Mujoco.

[Moguri]

@jratcliff63367 It really seems like you're making your own physics standard here with the plan to eventually write a glTF extension to handle that standard. While I would still prefer to collaborate, I also prefer the smaller, more focused scope of the currently proposed BLENDER_physics extension. Such an extension is easier to support (both by exporters and importers), which helps improve adoption. BLENDER_physics is also usable now (including one known exporter and one known consumer) while this extension still appears to have a fair amount of work left.

As such, I plan to continue to develop the BLENDER_physics extension. Perhaps your extension will eventually come to replace it as a more complete physics representation. The two extensions may also come to exist side-by-side as a simple option and a complete option. Hopefully there will still be opportunity to collaborate on physics in the future.

[jratcliff63367]

I was trying to create a completely generic representation of rigid body physics. If what I have in that spreadsheet doesn't meet that goal, then I have failed. All concepts in there are very generic and should work with most major physics engines; at least the ones I have used which are PhysX, Bullet, Newton, and Havok.

I am not trying to 'create my own'. My goal was to create a generic rigid body DOM which could be used by GLTF. If I have somehow failed in that, I'm not sure where. That spreadsheet is quite basic and simple at this point.

[Moguri]

@jratcliff63367 Maybe I didn't explain myself accurately, but "create a generic rigid body DOM which could be used by GLTF" is what I was getting at when I said you were creating your own standard. This could be it's own thing (i.e., a stand-alone physics description format), and it feels like it already is with XML, protobuf, and JSON implementations. I feel glTF is just acting as a wrapper to this format. I find the DOM you have so far to be overly generic and too inclusive for a lot of cases. A complete spec like this is great for some use cases and useful to have (as @meshula mentioned), but I believe a simpler spec also has its uses (easier to implement, still covers a lot of cases).

As a separate conversation, I noticed your spec does not have a way to lock motion on particular axes. Bullet calls this linear factor and angular factor, and may be useful to include.

[jratcliff63367]

PhysX supports locking motion as well, but I didn't think that was a generic concept. Every object in the spec is a 'Node' and every node can have a set of 'additional properties'. The idea is that engine specific concepts can be expressed under the additional properties fields. That said, if locking motion on an axis is a generic enough concept, it could be added to the base spec.

To clarify what my task at hand is. For an internal project at work I need a way to express rigid body physics in a generic way that can be run on multiple physics engine backends.

I believed this was also a goal of the GLTF project, so I felt there was an opportunity to 'kill two birds with one stone', since the goals seemed largely the same.

I do not think my current spec is complicated, in fact I thought it was quite simple and generic, but capable of expressed many complex rigid body systems.

If there is no desire for this effort to be integrated into GLTF, I can stop contributing to this effort here. My tasks for my work project remain the same.

[donmccurdy]

I believed this was also a goal of the GLTF project, so I felt there was an opportunity to 'kill two birds with one stone', since the goals seemed largely the same ... I do not think my current spec is complicated, in fact I thought it was quite simple and generic, but capable of expressed many complex rigid body systems.

In my opinion @jratcliff63367's proposal is both simple and generic for the expressed goal of providing a superset of physics engines' capabilities. In contrast, glTF has so far taken a "subset" approach — as one example, subject to some debate, we've limited assets to 4 influences per vertex for skinning, because some key engines cannot support more. This has the advantage that glTF assets can be expected to work very broadly, but is clearly less expressive.

Whether that same "subset" approach is right for physics, I don't know. There are features described here that cannot be expressed in JS physics engines like Cannon.js or Oimo.js, but JS frankly lags behind on good physics implementations, and I don't really expect that to change until our browsers' GPU APIs improve.

In my opinion that is the crux of the difference between the BLENDER_ proposal and this one. Having both may be a reasonable compromise. Or perhaps there is some way for one to "extend" the other? I'm not quite sure how to handle the protobuf and XML requirements, and would defer to you on that.

The alternative is that some things may be expressed as higher-abstraction wrappers around glTF, as 3D Tiles do to combine geospatial metadata and glTF assets, rather than as extensions.

[Moguri]

@jratcliff63367 Just to be clear, I am not saying that there is not a place for this extension. I am just one voice, and I find a more limited physics subset to work better for my current needs. I am sure other people would love a more complete physics spec. Also, I would not say that your spec is complicated; it just includes more (e.g., joints) than engines may want to (or even can) implement. I would say "complete" or "comprehensive" is a better description than "complicated."

[vpenades]

I would like to give my humble opinion about rigid body support on glTF;

I agree with @meshula in that maybe trying to add any kind of rigid body representation to glTF might go beyond what's really needed. Sometimes we must remember the KISS philosophy. Good things are tiny.

Let's think in terms of Model-View-Controller; Everybody would consider having "Employee" profile information embedded within the photography of that employee as a crazy design.

That being said. Having rigid body information embedded in the glTF file makes more sense from the point of view of how current artistic pipelines work today: I can see a 3D artist creating a 3D model, and including a rigid body layer so the artist can export everything in a single shot.

Another matter is whether to include the rigid body information within the gltf schema or not.... indeed, I would preffer to have everything in a separated schema AND a separated file.

Consider a physics engine having to load a single RigidBody+Visuals file, it needs to deal, and skip all the visual/graphic elements. The same happens for the graphics engine, it needs to skip the rigid body information data.

With separated files, each domain would load exactly what it needs, without interferences from the other.

[pjcozzi]

@jratcliff63367 @Moguri since I do not have the physics domain knowledge, I am relying on you two to guide this.

Based on @donmccurdy's comment, it sounds like it could be fine to have a very simple Blender physics extension and the full-featured extension - if the former isn't easily designed as a subset of the later, or the former can be done much more quickly. Does this make sense or not?

It's not a perfect analogue, but perhaps this is like glTF common materials vs. PBR. Both are valuable and expected to be widely implemented.

[pjcozzi]

@jratcliff63367 @Moguri we discussed this in the Khronos 3D Formats working group. The group is OK with having two extension, especially this early in the glTF physics ecosystem where we need to get more implementation experience to flush things out - it could benefit from different approaches.

Let me know if there is anything I or the group can to do facilitate things moving forward.

[JoshKlint]

The appeal of this to me is my users could set up collision shapes in a modeling program and export them in GLTF, giving them finer control than the auto-generated shape options we support in our editor.

This extension should be focused purely on GEOMETRY, for the artist's purposes. Joints, physical properties, and collision behavior definitions should definitely be kept out. If this is to be useful at all I think it needs to just define a physical shape (procedural or polygon / hull) for the whole object. Otherwise it turns into a physics engine serialization system.

In fact, the reason there is no standard physics shape format is likely because people just can't help themselves from expanding it into joints, physical properties, etc.

We use these shapes:

• box
• sphere
• cylinder
• capsule
• convex hull (standard GLTF mesh with positions only)
• polygon mesh (standard GLTF mesh with positions and indices only)
• cone
• chamfer cylinder
• compound (aggregate of any of the others)

If there are other shapes in other physics engines please list them.

I would leave heightfields out, because there are too many options for how those can be stored and used, and terrain is likely to be an engine-specific feature anyway.

For each shape, you need an orientation defined, so the standard mat4 or quat / position works fine. Is scale allowed?

Are we defining a collision shape for the entire GLTF scene, or do you want to define shapes on a per-node basis?

Do convex hulls need indice info or is a cloud of vertices enough?

[JoshKlint]

Here's a sample of what it might look like:

collisions {
    {
        "shape:" "BOX",
        "translation": [4,6,2]
    }
    {
        "shape:" "CONE",
        "rotation": [-0.70710678118654746,-0,-0,0.70710678118654757],
        "scale": [2,2,2],
        "translation": [1,0,0]
    }   
    {
        "shape:" "CONVEXHULL",
        "scale": [1,1,1],
        "translation": [1,0,0],
        mesh: 3
    }
    {
        "shape:" "COMPOUND",
        "collisions": [0,1,2]
    }
}

[JoshKlint]

Since GLTF is a scene file format, even though I use it exclusively as a model format, it makes sense that different objects in a scene would have different collision shapes. Therefore I suggest adding a "collisions" or "shapes" tag to the node structure that indicates one or more physics shapes:

nodes {
    {
        ...
        collisions: [0,3,6]
    }
}

This design would eliminate the need for compound / aggregate shapes, which would get unnecessarily complicated if they were nested.

[JoshKlint]

I'm coming from Newton, but the features of Blender largely line up: https://docs.blender.org/manual/en/latest/physics/rigid_body/properties.html

• Box
• Sphere
• Capsule
• Cylinder
• Cone
• Convex hull
• Mesh

Again, I would just focus on the collision geometry, because physical properties like mass can easily be typed in an engine's editor, and the reusability there is pretty much non-existent.

Just focusing on the above, one question that needs to be addressed is whether a sphere can have a non-uniform scale, or does it just accept a single radius value? Newton supports this but reading the above, it looks like Blender does not. http://www.newtondynamics.com/wiki/index.php/NewtonCreateSphere

Same question applies to cylinders, cones, and capsules (on two axes). Should it be something like this?:

    {
        "shape:" "CONE",
        "height": 2.0,
        "radius": 1.0
    }

Perhaps the scale should be stored the same way as in nodes (scale property or in matrix) but it should be specified that only the first component will be used in case of a sphere, and only the XY components will be used for a cylinder's diameter and height (if non-uniform scales are not to be supported). That takes care of it, while allowing for future expansion or wiggle room for those who choose to veer outside the spec:

    {
        "shape:" "CONE",
        "scale": [2.0, 2.0]
    }

What is the appropriate way to store a sphere?

    {
        "shape:" "SPHERE",
        "radius": 0.5
    }

    {
        "shape:" "SPHERE",
        "scale": [0.5]
    }

    {
        "shape:" "SPHERE",
        "scale": 0.5
    }

I lean towards making a firm rule that non-uniform radii are never allowed. GLTF seems to be at its best when its limitations are embraced, and I don't think I have ever encountered a situation where it was actually needed in all my years of engine development.

[Moguri]

@JoshKlint I plan to revive #1104 as a collisions-only extension and strip out rigid body properties (e.g., mass) for another, separate extension. If you're willing to collaborate, I can prioritize getting a draft of that extension published again.

[JoshKlint]

I am willing to help in any way. I don't have any experience writing extension specs but I am interested in putting some effort into it.

[JoshKlint]

Docs from 3ds max, looks like it aligns pretty well: https://knowledge.autodesk.com/support/3ds-max/learn-explore/caas/CloudHelp/cloudhelp/2019/ENU/3DSMax-Simulation-Effects/files/GUID-B0C87F87-6E4B-41C4-B0EB-94ACE223E9C9-htm.html

[JoshKlint]

Drilling down into this, an array of positions isn't really a complete description of a convex hull at all. Blender has a minimum triangle area value, and Newton has a tolerance value used for vertex merging: http://newtondynamics.com/wiki/index.php/NewtonCreateConvexHull

The point is, a list of vertices is not sufficient to create a consistent reproducible convex hull. A real definition of a convex hull would consist of a vertex array and an array of faces / polygons with a variable number of sides. So should the hull construction step take place before the hull is saved in the file, or after the raw data is loaded from the file?

Each face could be defined as a plane equation, or as a list of indices into the vertex array. If a list of indices are provided, the plane equation would be redundant because it can be calculated from the first three vertex positions. That might mean that this extension is dependent on completion of the n-gon stuff others are defining (https://github.com/KhronosGroup/glTF/pull/1620). Is a GLTF mesh really appropriate to try to cram into a convex hull definition at all?

I'm afraid if a full definition is not provided then a lot of extra values will be tagged in like "Blender minimum triangle area" or "Newton tolerance" leading to inconsistent results.

I'm sure many implementations, including our own, will initially ignore the indice list and reconstruct the hull from the vertex list, but I think things should move towards an unambiguous definition of a convex hull.

[drichardson]

@JoshKlint As you point out, there are multiple ways to generate a convex hull and some of them are slow (e.g., Unreal Engine's convex hull generator in the static mesh editor), and also don't produce good enough hulls in some situations. The first paragraph of the glTF website states:

glTF minimizes both the size of 3D assets, and the runtime processing needed to unpack and use those assets

Given the ambiguity and performance issues, the convex hull construction should be done before it is saved to the glTF file.

[Moguri]

Maybe it's best to leave convex hull out of the initial extension and leave it for a version 2 of the extension (or possibly another extension)?

[drichardson]

@Moguri I'm using Blender/Houdini to author models, and generating collision geometry. If I wasn't able to use convex hulls, it would be near impossible to get decent collision geometry (other than just using the mesh itself).

I don't think a glTF specification needs to define how a convex hull is generated. Rather, it only needs to allow one to be specified.

[JoshKlint]

Typically a series of convex hulls is the only way to define a concave or complex physics shape. Unreal's convex hull generator is probably doing convex decomposition, which is much more inexact than a single hull...it is turning a polygon mesh into multiple convex hulls that estimate the original shape.

A convex hull in GLTF would be a single convex solid. This is nice because you can model the exact shape you want, using one box or whatever for each concave piece. Sometimes that is the only way to get the right shape in Leadwerks, as convex decomposition can sometimes produce very rough results. Single hull generation should be a fast process that can be performed at load time.

I don't want to exaggerate the problem of describing a single convex hull as a vertex cloud, but there are some situations down the line where it could fail, and if we are locking it into the spec we should try to anticipate those problems, even if individual loaders just generate a new hull from the points. A simple vertex cloud would probably be fine >95% of the time.

Maybe a hull would look something like this:

{
    "shape:" "CONVEX_HULL",
    "points": [0.0,0.0,0.0, 0.0,0.0,1.0, 1.0,0.0,1.0, 1.0,1.0,1.0, 2.0,3.0,4.0, 5.0,6.0,7.0],
    "faces": {
        {
            "indice_count": 4,
            "indices": [0,3,4,1]
        },
        {
            "indice_count": 4,
            "indices": [0,2,4,1]
        },
        {
            "indice_count": 4,
            "indices": [2,5,4,1]
        },
        {
            "indice_count": 4,
            "indices": [1,0,2,3]
        }
    }
}

These are typically very simple geometry, so a purely text-based definition might be okay. If you have a very complex convex hull, that is generally a problem.

So I see three possible ways to define hulls:

• Ignore potential problems with ambiguity and just store them as a series of vertices in a mesh.
• Store them in a more explicit text-based definition that includes face definitions, as above.
• Store them in more explicit buffers that includes face definitions, which could make complex hulls faster-loading, even though for the intended usage complex convex hulls shouldn't really be happening.

I might be overthinking the whole thing, and we can just supply the verts and let the individual implementations worry about the details of creating the hull. Newton, Bullet, and PhysX all require a vertex cloud as input and have a "wiggle room" variable to generate the hull, so it is likely these would continue to just fall back on generating a new hull from the points, rather than building a hull according to the exact definition the file contains: http://newtondynamics.com/wiki/index.php/NewtonCreateConvexHull https://docs.nvidia.com/gameworks/content/gameworkslibrary/physx/guide/Manual/Geometry.html http://www.staff.city.ac.uk/~andrey/INM377/bullet-2.82-html/html/classbtShapeHull.html#a21a86945965e1cedf7b257f2882d6e54

[JoshKlint]

Here's a potential solution that keeps things simple but gives us an escape hatch if there are problems in the future:

• A convex hull is defined by a mesh that is required to contain vertex positions.
• An optional indice array is allowed in the mesh, to indicate visible polygons.
• If the indice array is stored using the FB_ngon_encoding extension, then the exact faces of the hull are specified.

The ideal situation would be that the exporter save the data with an indice array encoded with the FB ngon scheme, and the loader recognize and load the same thing, but the simple vertex cloud is what will probably end up getting used most of the time.

[JoshKlint]

The FB_ngon_extension could also be useful for the polygon mesh collision shape, at least in Newton. It allows faces with any number of verts, and I think this can reduce problems with collisions along triangle edges: http://www.newtondynamics.com/wiki/index.php/NewtonTreeCollisionAddFace

[vpenades]

@JoshKlint Although I agree that some kind of collision system is needed, a recurrent issue I see with having both physics and visuals in the same file is domain separation.

Not everybody uses Unity or Unreal, which use glTF as an authoring format. We use glTF as a "last mile" format, loaded at runtime.

In our case we have a customized engine based on MonoGame, where physics and visuals are completely separated. That means the graphics system is responsible of loading the visual representation of a model, and the physics engine loads the physical representation.

The loading is done on demand, which means the physics and visuals loading is done at different times.

We also have a testing backend that only tests the physics side of things, so the visuals side is not needed at all, in fact, the physics library is pure code, with no graphics dependencies. So if we had to allow the physics engine to load glTF files that contain visuals representations, we would have to make the physics engine depend on graphics/visuals engine libraries, which is nasty.

So in our case, it would force us to have domain mixed dependencies, and we would have to load the whole file two times.

[JoshKlint]

Not everybody uses Unity or Unreal, which use glTF as an authoring format. We use glTF as a "last mile" format, loaded at runtime.

I am planning to do the same in my new engine. I think loading just the collision geometry from a GLTF file would be very fast if you skipped the graphical bits, but it's certainly not something everyone has to use.

Perhaps a standalone GLTF file containing just a physics body definition would be useful to you in the future.

[icavalheiro]

@vpenades , I agree with @JoshKlint . This is just a development approach you would have to handle for your game engine. Maybe you don't have to depend on the the graphics engine to load your physics from GLTF, you can just create a common dependency (GLTF loader) that handles both and make your graphics and physics engines depend on it. That way your physics engine can "care" about the physics parts only and use that a build/test time, while the graphics would handle it's part on run time. Anyway, I don't see this as being an issue with the file format. I can only see benefits from GLTF handling rigid bodies, specially for browser engines.

[vpenades]

@icavalheiro you miss the point... of course we can add physics to glTF, as we can add Midi partitures or entire PDF documents if we wish. I'm not talking whether we can add physics to glTF. My question is... SHOULD we add physics to glTF?

My opinion is NO, glTF is a graphics domain, as in JPG and PNG, it's a beast on its own, and very optimized to be a last mile graphics file format.

Physics is a completely different monster that requires its own set of domain specific, optimized structures that in some cases might fit into glTF and in some others might be totally incompatible, which will lead to a very and unnecessarily convoluted file format.

Furthermore, I fear the added complexity of physics might compromise the development of additional graphics extensions to the format.

In these cases I always like to remind people the case of the Collada file format; a format that wanted to be too many things, and ended being a niche format.

In terms of complexity, I preffer to follow the KISS philosophy, do one thing, and do it right... so if a physics standard is to be defined, I would preffer to have it in its own domain, schema, and namespace; a while ago I proposed plTF.

[Leadwerks]

I have written a draft here of a spec that handles collision shapes: https://github.com/Leadwerks/glTFExtensions/blob/main/ULTRA_collision_shape/README.md

The main purpose of this is to simplify the art pipeline so that artists I work with can set up collision shapes in Blender and have those load into our software.

It covers the main features of Blender, Newton Dynamics, and Bullet Physics. Convex hulls are explicitly defined, and stored in binary. Polygon mesh shapes support a varying number of indices per face, or can use a constant number of indices (i.e. triangle or quad meshes). Multiple shapes can be defined per node to form a compound shape.

Physics properties such as mass, and joints/constraints are outside the scope of this extension.

[vpenades]

@Leadwerks have you seen this? https://github.com/Kupoman/blendergltf/tree/master/extensions/BLENDER_physics

[Leadwerks]

@vpenades Looks okay, but I believe the convex hull should have a precise definition of the geometry, not just a vertex cloud. Also, in Newton each face in a collision mesh can also have a variable number of indices, and it does make a real difference in the behavior. A quad will give better behavior than two triangles, for example, under some conditions.

I guess it's just a matter of who wants to do the work adding this functionality to the Blender exporter? Maybe we should be talking about that.

[jamesdolan]

FWIW there is a new effort to do this... https://www.khronos.org/assets/uploads/developers/presentations/Physics_BOF.pdf https://github.com/eoineoineoin/glTF_Physics

[jratcliff63367]

When is there not always a new effort to do this? FYI, there is a new convex decomposition library called CoACD (for collision aware convex decomposition) which appears to work very well and I recommend it.

[npolys]

There is an ISO-IEC specification (X3D) that specifies a proven and engine-agnostic for Rigid Body Physics in interactive 3D applications : https://web3d.org/documents/specifications/19775-1/V4.0/Part01/components/rigidBodyPhysics.html

We should consider harmonizing these!