mariuszhermansdorfer / SandWorm

Augmented Reality Sandbox for Grasshopper & Rhino
MIT License
20 stars 11 forks source link

Intelligent hand masking/removal #31

Closed philipbelesky closed 2 years ago

philipbelesky commented 4 years ago

This might be stretching the limits of what is possible to do within the performance budget of a definition, but other sandbox style applications seem to intelligently 'pause' updating a mesh when users are reshaping the sand with their hands. This gets around some of the oddities that come with the mesh suddenly updating to incorporate hands as if they were part of the landform.

The Kinect has APIs for hand/limb detection, so one way to go might be divide the pixels into a clustered grid (via an R-tree?) and skip updating those depth pixels as long as there is a hand present within the grid.

mariuszhermansdorfer commented 4 years ago

A quick & dirty way of reducing the artifacts would be to change the arbitrary value in this condition

And set it to ignore, say the first 700 mm as seen from the sensor. Current logic would sample the value from one of the neighbors but it could be set to recalling its previous state.

This approach wouldn’t work, however, when hands would be operating within the range of the actual terrain.

If we were to go with hand recognition and removal, I’d like to keep it as an option. We sometimes use the box to visualize contour lines on complex surfaces which we create via bending paper sheets. It gives students a good idea on how contour lines flow along these otherwise hard to comprehend shapes.