google-ai-edge / mediapipe-samples

Apache License 2.0
1.62k stars 416 forks source link

facialTransformationMatrixes not work #293

Open 1101728133 opened 11 months ago

1101728133 commented 11 months ago

image

PaulTR commented 11 months ago

Can you share a trimmed down version of your project? The print looks like it's pointing to the float array that's stored in the object. Maybe try printing [0] from there - https://developers.google.com/mediapipe/api/solutions/java/com/google/mediapipe/tasks/vision/facelandmarker/FaceLandmarkerResult#public-abstract-optionallistfloat[]-facialtransformationmatrixes

1101728133 commented 11 months ago

你能分享一下你的项目的精简版本吗?打印看起来指向存储在对象中的浮点数组。也许尝试从那里打印 [0] - [https://developers.google.com/mediapipe/api/solutions/java/com/google/mediapipe/tasks/vision/facelandmarker/FaceLandmarkerResult#public-abstract-optionallistfloat[]-面部转换矩阵] (https://developers.google.com/mediapipe/api/solutions/java/com/google/mediapipe/tasks/vision/facelandmarker/FaceLandmarkerResult#public-abstract-optionallistfloat%5B%5D-facialtransformationmatrixes)

https://github.com/googlesamples/mediapipe/blob/main/examples/face_landmarker/android/app/src/main/java/com/google/mediapipe/examples/facelandmarker/fragment/FaceBlendshapesResultAdapter.kt

 fun updateResults(faceLandmarkerResult: FaceLandmarkerResult? = null) {
        categories = MutableList(52) { null }
        val shapeMap = mutableMapOf<String, String>()
        val newshapeList = mutableListOf<String>()
        if (faceLandmarkerResult != null) {
            val sortedCategories =
                faceLandmarkerResult.faceBlendshapes().get()[0]
            val min = kotlin.math.min(sortedCategories.size, categories.size)
            for (i in 0 until min) {
                if (i != 0) {
                    categories[i] = sortedCategories[i]
                    shapeMap[sortedCategories[i].categoryName()] =
                        String.format("%.2f", sortedCategories[i].score())
                }
            }

            for (i in FaceBlendShape){
                newshapeList.add(shapeMap[i].toString())
            }
            newshapeList.add("0")
            newshapeList.add("0")
            newshapeList.add("0")
            println(faceLandmarkerResult.facialTransformationMatrixes().get()[0])
            var ip = getGlobalVariable().toString()
            UDPManager.SendMess(newshapeList.toString(), ip, 11112)
            newshapeList.clear()
        }
    }

<faceLandmarkerResult.faceBlendshapes().get()[0]>work,but <faceLandmarkerResult.facialTransformationMatrixes().get()[0]>not work. Here, I need to sort the values of blendshape in standard order and add the Euler angles obtained from the matrix to the end. Python works, but this doesn't work

1101728133 commented 11 months ago

Are you still there

PaulTR commented 7 months ago

I am now :P Sorry went on leave a bit earlier than expected, but returning now (technically tomorrow) and following up on some of the things that fell through the cracks. I'll have this on my review list.

Mehrdadjdev commented 4 months ago

any updates?