Get a Face Texture with the Face Mesh in ARKit?

[yosun]

How do I Get a Face Texture with the Face Mesh in ARKit?

Corollary:

• Does TryGetFaceMeshUVs() return different results or is UV mapping the same for all ARKit faces?
• Is there a TryGetFaveMeshTexture() ?

[DavidMohrhardt]

There is no TryGetFaceMeshTexture as ARKit does not supply any texture with the ARFaceAnchor member ARFaceGeometry.

You might be able to accomplish this using a GrabPass Shader.

[yosun]

Ok, but does TryGetFaceMeshUV ever return a different UV map? It seems that the face masks are all mapped the same, with the same vertices, etc?

[DavidMohrhardt]

No it should not. That method is a way to acquire the values from the native ARKit ARFaceGeometry instance property textureCoordinates.

The relevant section in the documentation:

Each float2 value in this array represents the UV texture coordinates 
for the vertex at the corresponding index in the vertices buffer.Face 
mesh topology is constant across ARFaceGeometry instances, so the values 
in this array always maps the same vertex indices to the same texture 
coordinates.

So you should be able to call TryGetFaceMeshUV once per application run and the texture coordinates will be the same for the all instances of face geometry.

[yosun]

How would you grab the face mesh texture if it's just once per detected face?

[DavidMohrhardt]

I don't quite understand what you mean. You would need to grab a different texture per face but the actual UV mapping to the vertices wouldn't change between the faces. Essentially you'd need to instruct the user to align their face with the camera and make sure the face is taking up a decent amount of the screen (so the greatest amount of face is visible) then grab that section of background texture which could be done using the aforementioned Grab Pass Shader Technique.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[UnityEQ]

very little on google for "TryGetFaceMeshUV."

get the real life Face Texture of a person so I can apply that texture to a Face Mesh in ARKit(arfoundation)? I would think a very specific shadergraph would do this. An example from top-men would be appreciated.

[yosun]

very little on google for "TryGetFaceMeshUV."

get the real life Face Texture of a person so I can apply that texture to a Face Mesh in ARKit(arfoundation)? I would think a very specific shadergraph would do this. An example from top-men would be appreciated.

did u try it