Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
None
OS Platform and Distribution
Android
MediaPipe Tasks SDK version
Vision 0.10.14
Task name (e.g. Image classification, Gesture recognition etc.)
FaceLandmarker
Programming Language and version (e.g. C++, Python, Java)
Kotlin
Describe the actual behavior
I am currently using the MediaPipe Task Vision's FaceLandmarker to detect faces in my Android application. I can successfully get the FaceLandmarkerResult, but I am unable to access FaceGeometry from it. My goal is to extract the 3D mesh data, specifically the VertexBufferList, for further processing.
Describe the expected behaviour
I expected to retrieve the FaceGeometry from FaceLandmarkerResult and then access the mesh data through getMesh() and getVertexBufferList(). This should allow me to work with the 3D landmarks in a structured manner, which is essential for my application's functionality.
Standalone code/steps you may have used to try to get what you need
Below is the code snippet I used:
private fun loadModels() {
val canonical = ModelRenderable
.builder()
.setSource(this, Uri.parse("models/canonical.glb"))
.setIsFilamentGltf(true)
.setAsyncLoadEnabled(true)
.build()
try {
canonicalNode = Node()
canonicalNode.setRenderable(canonical.get())
canonicalNode.localScale = Vector3(10f, 10f, 10f)
val rotation = Vector3(1f, 1f, 1f)
canonicalNode.localRotation = Quaternion.axisAngle(rotation, 0f)
canonicalNode.localPosition = Vector3(0f, 0f, -4f)
transparentSceneView.scene.addChild(canonicalNode)
} catch (ignore: InterruptedException) {
} catch (ignore: ExecutionException) {
}
null
}
}
private fun updateThePosition(
faceLandmarkerResult: FaceLandmarkerResult,
) {
val faceLandmarks = faceLandmarkerResult.faceLandmarks()[0]
// i can get the 468 face landmarks.
//val index = 6 //landmark index 6 for on the nose
//val faceGeometry = faceLandmarkerResult.faceGeometry
//val mesh = faceGeometry.getMesh()
//val vertex = mesh.getVertexBufferList()
canonicalNode.localPosition = Vector3(vertex[index * 5] , vertex[index * 5 + 1], vertex[index * 5 + 2])
}
However, the FaceGeometry and methods like getMesh() or getVertexBufferList() do not seem to be accessible from the FaceLandmarkerResult or its nested classes.
Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
None
OS Platform and Distribution
Android
MediaPipe Tasks SDK version
Vision 0.10.14
Task name (e.g. Image classification, Gesture recognition etc.)
FaceLandmarker
Programming Language and version (e.g. C++, Python, Java)
Kotlin
Describe the actual behavior
I am currently using the MediaPipe Task Vision's FaceLandmarker to detect faces in my Android application. I can successfully get the FaceLandmarkerResult, but I am unable to access FaceGeometry from it. My goal is to extract the 3D mesh data, specifically the VertexBufferList, for further processing.
Describe the expected behaviour
I expected to retrieve the FaceGeometry from FaceLandmarkerResult and then access the mesh data through getMesh() and getVertexBufferList(). This should allow me to work with the 3D landmarks in a structured manner, which is essential for my application's functionality.
Standalone code/steps you may have used to try to get what you need
However, the FaceGeometry and methods like getMesh() or getVertexBufferList() do not seem to be accessible from the FaceLandmarkerResult or its nested classes.