[proposal] Use meshes for object selection

[Desour]

Problem

To aim at / select an object, you have to aim at its selectionbox. The selectionbox however is very imprecise in most cases, due to objects not being cuboids, and their rotation and animation. This leads to sheep being selected when you aim through the gaps between their legs and not being selected when you aim through their head. Also, when the object punched or "rightclicked", the mods don't know exactly which piece of the object was clicked. This is a limitation of the api. (Possible usecases are listed below.) Note: The focus of this feature request is not on the visuals of the box displayed when an object is selected, nor on the collisionbox. It's about the player aiming at objects (Edit: and nodes) and clicking on them.

Solutions

• I'd like to being able to define a selection-mesh in the object properties, that is used by the client for ray-mesh collision testing.
• Coarse ray-cube, ray-cuboid, ray-ellipsoid or ray-sphere collision testing with an outer box / sphere / whatever can be used to keep performance good. This outer shape could be defined in the object properties, or gotten automatically by analysing the animations and all (which sounds complex, so former option will probably be preferable).
• Most objects will want to use their visual mesh as selection-mesh, so there should maybe be a shortcut for this.
• When the object is selected, we can still show the normal selection box.
• The selection mesh should be rotated and animated exactly like the visual mesh.
• There could be a texture that is sampled to check where the selection mesh is through-click-able or solid.
• Api callbacks (ie. on_punch) can receive more details about the click location:
  • absolute exact 3d click location (ray collision point): for particles and positional sounds
  • relative exact 3d click location: for attached particle spawners (ie. a bleeding wound) and attached sounds (ie. gas leak)
  • exact 2d click location according to the mesh's uv-map: for weak spots of mobs; and for different sounds depending on punched material (ie. punch car wheel rubber vs. punch car metal)
  • index of clicked triangle: to make hitting a mob at the same position less or more efficient (the 2d uv pos is not enough: consider a mob that gets blind if both eyes are stabbed out, but the eyes are possible uv-mapped to the same eye in the texture)
  • material and other things defined by mesh
• As many mods won't need all of the details, there could be an object property that tells which things the client should send.

Alternatives

(Late edit:) In some cases, i.e. if you want to know what body-part of an enemy you punched, it might be a good option to use convex 3d-polytopes:

• Define the polytopes via oriented planes. (A polytope is the intersection of all the half-spaces defined by this planes.)
• Also bind each polytope (or each plane) to a bone.
• It's easy to check if the camera is in a polytope (dot-product with each plane, then sign-check).
• If the camera is not in the enemy, you can make ray-plane intersection with each plane of each polytope. If it is inside all the other half-spaces of the polytope, it is a hit. And there's always exactly one forward-face hit and one backward-face hit, or no hit.

Pros:

• It's easy to tell which body parts were hit, even when you're inside.

Cons:

• Needs many polytopes to be exact (e.g. mtg boat).
• Is much more difficult for modders (i.e. harder to derive from visual mesh, and can't create with modelling software).
• Non-rigid animation might cause issues.
• The mesh-based solution can probably do the same with faces inside the mesh (though z-fighting might be an issue).

Additional context

The implementation effort for most of this is not unrealistically high. I've actually some time ago implemented the very basic features of this request using irrlicht's collision stuff: https://github.com/Desour/minetest/commit/93166d30779641d26bc6e54692bd8777aaef8d71 But as the unused parts of irrlicht are removed now, it probably won't compile anymore. (It had some problems with animation and stuff anyway.)

Here's a screenshot:


[AFCMS]

* Api callbacks (ie. `on_punch`) can receive more details about the click location:

  * absolute exact 3d click location (ray collision point): for particles and positional sounds
  * relative exact 3d click location: for attached particle spawners (ie. a bleeding wound) and attached sounds (ie. gas leak)
  * exact 2d click location according to the mesh's uv-map: for weak spots of mobs; and for different sounds depending on punched material (ie. punch car wheel rubber vs. punch car metal)
  * index of clicked triangle: to make hitting a mob at the same position less or more efficient (the 2d uv pos is not enough: consider a mob that gets blind if both eyes are stabbed out, but the eyes are possible uv-mapped to the same eye in the texture)
  * material and other things defined by mesh

I also would like to add this as an option to raycast (nodes+entities). This would really be helpful for raycast based weapons, especialy for graphics, headshots, etc

[Desour]

[...]

I also would like to add this as an option to raycast (nodes+entities). This would really be helpful for raycast based weapons, especialy for graphics, headshots, etc

(I guess you're referring to the raycast thing in the server-side lua api.) While I absolutely agree that this would also be useful for those server-side raycasts, it might be more complicated to implement because AFAIK the server currently does nothing with meshes, ie. doesn't even read the mesh media files.

[AFCMS]

And how about adding this setting, but only for CSM/SSCSM? Would be roughly the same for my personal usage

[Desour]

For CSM/SSCSM my comment from above of course doesn't apply.

[appgurueu]

I have managed to implement this serverside using only a mod, implementing - among other things - a way to resolve model paths, a B3D model reader and ray-triangle intersections; I still haven't gotten to adding animation support. This would be a good feature to have in the engine.