search

cabanier / webxr-face-tracking-1

13 stars 0 forks source link

Add head pose for phone-based usecases #2

Open hybridherbst opened 1 year ago

hybridherbst commented 1 year ago

For Quest Pro and similar "inside facing" devices I see how only the facial blendshapes are needed, because head pose equals device pose.

However, for phone-based usecases such as face tracking on Android or iOS, an additional head pose (pos, rot, scale) is needed to match from screen-based session to a reasonable head model.

cabanier commented 1 year ago

Maybe we can expose something like the body API which returns the positions of head, torso, arms, etc?

hybridherbst commented 1 year ago

In practical terms APIs (ARKit / ARCore) nowadays usually let the developer choose between a "fully-body mode" (no face blendshapes) and "face mode" (limited body, usually only head). Not sure how feasible mixing everything into one proposal would be.

cabanier commented 1 year ago

I think they would have to be separate specs. Body would contain the person's physical characteristics so we have to be very careful when exposing those.