Mesh colliders - Githubissues

erikfrey commented 2 years ago

More "real" simulations often involve complex collider geometry that must be modeled accurately in the simulator.

Convex hull is probably sufficient. Would need to support not just Mesh<>Mesh, but also Mesh<>Capsule, Mesh<>Plane, ...

erikfrey commented 2 years ago

As of e2ab3aaa84f1d6ae1280ec79c5d77fb44e9c3777 we now have initial support for mesh colliders, but right now have only implemented mesh<>plane and mesh<>capsule. Check it out!

Before we call this done we'd also like to add mesh<>mesh and also some example documentation for how to use it, perhaps include it into the colabs.

But for now, here's the Utah teapot colliding with a plane and a capsule in Brax:

utah_teapot

EelcoHoogendoorn commented 2 years ago

Interesting. Ive wondered how to elegantly implement this kind of thing in brax. Never tried my hand at it, but it occurred to me that deepSDFs might be the way to go here. Given a function to find intersection points between two SDFs, and some handcrafted SDFs for simple primitives, it should be easy to just fit SDFs to arbitrary meshes using established techniques, and you should have a very nice and generic framework, that should paralelize just fantastically on the TPU.

Some googling shows this hot off the press. Nothing remarkable here, just an implementation of the above. But the performance comparison is interesting. It seems to compare favorably in terms of compute/memory tradeoff; and I think the compute would look even better if youd consider it in the context of a parallelized simulation running many actors batched in the same env.

They dont seem to cover SDF-SDF collision handling though; I suppose that raises some questions about efficient sampling strategies of query points. If youve got some half-decent broad phase culling I suppose just brute-forcing all vertices wouldnt be half bad. But then what are the vertices of a capsule, or a sphere? Of course one can pre-sample a bunch of points; but it would introduce some discreteness. And what about edge-edge contacts? I suppose you could easily turn this topic into a whole phd thesis... but I do wonder if there isnt something quite simple and effective out there that would be 'good enough' for 95% of use cases; my gut feeling says yes.

EelcoHoogendoorn commented 2 years ago

Speaking to the notion of 'good enough'; seems like the current box-box collision implementation only handles box-corner contacts; and as such probably yolos the edge-edge contact scenario entirely (oh wait it does not do box/box at all it seems? same conclusion tho). Would be an interesting exercise to reimplement the box-box logic as more generic convex hulls, implemented as 'handcrafted deep SDFs'; a convex hull is just the max over a bunch of linear projections, after all. That would make improving edge-edge handling easy; just spawn some extra vertices on the edges.

Does feel a bit wrong to be generating query points and probing them vs an SDF when it comes to handling cases like sphere-sphere interactions. Perhaps there should be a system for generating query points; either they come from some static set in the case of a general mesh, but there could be a dynamic override function implemented for special cases, like sphere-sphere or what have you, in which case there are obvious analytical solutions. Might also be helpful in generating edge-edge contact points for cubes, for instance.

esculapa commented 2 years ago

Hi, can you share a code for the teapot or some other example of using meshes? I am trying to simulate lifting a box with outer boxes, but there are no box vs box colliders.