Tastenkunst / brfv5-browser

Beyond Reality Face SDK - BRFv5 - Platform: Browser
229 stars 48 forks source link

magic numbers when mapping to 3d? #9

Closed aferriss closed 4 years ago

aferriss commented 4 years ago

I'm looking through the threejs__brfv5_mapping.js file and wondering if you could explain exactly how and why the translation and rotation values are calculated the way they are. Things seem to work, but I'd like a better understanding of the math that is happening there.

Specifically, how is the z value chosen? It's written as

  let modelZ = 2725

  if(t3d.camera.isPerspectiveCamera) {
    modelZ = (2725 / 480) * (canvasHeight / t3d.sceneScale)
  }

I assume the 480 comes from the camera height, but what about the 2725?

Same thing with the face scale?

    let scale =   face.scale * si * 0.0133

Where does the 0.0133 come from?

Is there a better or simpler way to map the face xy and rotation into a three.js coordinate space?

MarcelKlammer commented 4 years ago

The facial data returned by BRF is more suitable for orthographic cameras. There is no z value, only scale. So when using the perspective camera, I needed to settle on a certain fov, so the bows look more or less correct and pointing towards the ears.

Once a fov was chosen, since we don't have z, I had to chose a fixed z plane value, which is 2725 for now for orthographic cameras and for perspective cameras a factor needs to be multiplied.

So instead of z, the scale value determines the size of the model.

That's all there is about that.

MarcelKlammer commented 4 years ago

And with that being said. You can now load your 3d model into the editor:

https://artov5.com/tpptv5/

and place it onto the head and it will fit once loaded into the example.

Hope that helps.