Expression tracking is a technology that reports a set of expressions of the current user. The expressions report how much a portion of the face or an eye move. They don't report actual positions or other real world data.
The main use case is to render the user's face and eyes using an avatar for increased social presence. This technology will NOT:
This API will define an extensive set of expression and will on a per frame basis, report which ones were detected and how strong they are. For instance, if the device's eye tracking is functional and the user looks left, it will return that both eyes are looking left.
enum XRExpression {
"brow_lowerer_left",
"brow_lowerer_right",
"cheek_puff_left",
"cheek_puff_right",
"cheek_raiser_left",
"cheek_raiser_right",
"cheek_suck_left",
"cheek_suck_right",
"chin_raiser_bottom",
"chin_raiser_top",
"dimpler_left",
"dimpler_right",
"eyes_closed_left",
"eyes_closed_right",
"eyes_look_down_left",
"eyes_look_down_right",
"eyes_look_left_left",
"eyes_look_left_right",
"eyes_look_right_left",
"eyes_look_right_right",
"eyes_look_up_left",
"eyes_look_up_right",
"inner_brow_raiser_left",
"inner_brow_raiser_right",
"jaw_drop",
"jaw_sideways_left",
"jaw_sideways_right",
"jaw_thrust",
"lid_tightener_left",
"lid_tightener_right",
"lip_corner_depressor_left",
"lip_corner_depressor_right",
"lip_corner_puller_left",
"lip_corner_puller_right",
"lip_funneler_left_bottom",
"lip_funneler_left_top",
"lip_funneler_right_bottom",
"lip_funneler_right_top",
"lip_pressor_left",
"lip_pressor_right",
"lip_pucker_left",
"lip_pucker_right",
"lip_stretcher_left",
"lip_stretcher_right",
"lip_suck_left_bottom",
"lip_suck_left_top",
"lip_suck_right_bottom",
"lip_suck_right_top",
"lip_tightener_left",
"lip_tightener_right",
"lips_toward",
"lower_lip_depressor_left",
"lower_lip_depressor_right",
"mouth_left",
"mouth_right",
"nose_wrinkler_left",
"nose_wrinkler_right",
"outer_brow_raiser_left",
"outer_brow_raiser_right",
"upper_lid_raiser_left",
"upper_lid_raiser_right",
"upper_lip_raiser_left",
"upper_lip_raiser_right"
};
interface XRExpressions {
iterable<XRExpression, float>;
readonly attribute unsigned long size;
float get(XRExpression key);
};
partial interface XRFrame {
readonly attribute XRExpressions? expressions;
}
This API is inspired by the OpenXR face extensions.
Face tracking exposes sensitive sensor data. Because of this, the user agent must ask the user for their permission when a session is asking for this feature (much like WebXR and WebXR hand tracking). In addition, sensitive values such as eye position must be rounded.