ShivanshuKantPrasad
commented
5 years ago This is done by doing a ray cast. Basically you create a ray from the mouse position towards screen and check what does that ray collide(intersect with). There may be other methods, but this is the one i know about.
Apologies if this concept has a particular name that I'm unaware of. What do I mean by the title? Suppose you have a 3D scene being rendered with several objects in it and you want to interact with them via your mouse cursor. How do you know what particular object you are clicking on?
To illustrate what I mean here is a simple 3D scene I made in p5js with a bunch of random boxes in it:
How to determine exactly what box I clicked on? Consider that there may be objets behind the box but we only want to interact with the foremost box. Also of consideration is the fact that the scene might not be static, that is, it might be rotated by any axis.
I needed this when I was experimenting with making a Rubik's Cube in p5js. I wanted to know where exactly I was clicking so that I could rotate a particular face of the cube.
In the end I came up with a solution that involved looking at the pixels and determine if my cursor was on a particular value (off by 1 from the rest of the cube, imperceptible to the human eye but an easy thing to do for a machine).
I know this only works because A) the cube faces are all solid colors and B) it has no lightning.
Here's a JSFiddle paste of the Rubik's Cube I made which, although it works, perhaps serves more as a what not to do example.
https://jsfiddle.net/fke831ap/embedded/
It also includes a custom loadPixels() function for WEBGL which, at the time, p5js didn't have.