Open jackbdu opened 6 months ago
There are two other keypoints which will be missing for the same reason.
Part: lips is missing indices: 291
Part: leftEye is complete.
Part: leftEyebrow is missing indices: 285
Part: leftIris is complete.
Part: rightEye is complete.
Part: rightEyebrow is missing indices: 55
Part: rightIris is complete.
Part: faceOval is complete.
It seems like the expectation is that the end of one pair will be the start of the next pair, so I ran a quick code to find instances where this is not the case. Not all of these "jumps" lead to missing indices. For the left/right eye the index which would potentially be left off (362
/ 133
) also appears at the end of the last connection so it gets included there. Same for index 308
in the lips.
Part: lips
Connection jump at index 10: 291 -> 61 between pairs [375,291] and [61,185].
Connection jump at index 20: 291 -> 78 between pairs [409,291] and [78,95].
Connection jump at index 30: 308 -> 78 between pairs [324,308] and [78,191].
Part: leftEye
Connection jump at index 8: 362 -> 263 between pairs [382,362] and [263,466].
Part: leftEyebrow
Connection jump at index 4: 285 -> 300 between pairs [295,285] and [300,293].
Part: leftIris
All okay
Part: rightEye
Connection jump at index 8: 133 -> 33 between pairs [155,133] and [33,246].
Part: rightEyebrow
Connection jump at index 4: 55 -> 70 between pairs [65,55] and [70,63].
Part: rightIris
All okay
Part: faceOval
All okay
Code I used for both checks:
const parts = {
lips: LIPS_CONNECTIONS,
leftEye: LEFT_EYE_CONNECTIONS,
leftEyebrow: LEFT_EYEBROW_CONNECTIONS,
leftIris: LEFT_IRIS_CONNECTIONS,
rightEye: RIGHT_EYE_CONNECTIONS,
rightEyebrow: RIGHT_EYEBROW_CONNECTIONS,
rightIris: RIGHT_IRIS_CONNECTIONS,
faceOval: FACE_OVAL_CONNECTIONS,
};
Object.entries(parts).forEach(([part, connections]) => {
const allIndices = MEDIAPIPE_FACE_MESH_KEYPOINTS_BY_CONTOUR[part];
const omissions = connections
.flat()
.filter((index) => !allIndices.includes(index));
if (omissions.length > 0) {
console.log(`Part: ${part} is missing indices: ${omissions.join(", ")}`);
} else {
console.log(`Part: ${part} is complete.`);
}
});
Object.entries(parts).forEach(([part, connections]) => {
const jumps = connections
.map((current, i) => {
const previous = connections[i - 1];
if (previous && current[0] !== previous[1]) {
return `Connection jump at index ${i}: ${previous[1]} -> ${current[0]} between pairs [${previous}] and [${current}].`;
}
return;
})
.filter(Boolean);
console.log(`Part: ${part}`);
if (jumps.length > 0) {
console.log(jumps.join("\n"));
} else {
console.log("All okay");
}
});
Hi, @jackbdu, @lindapaiste
We sincerely apologize for the delay in our response and thank you for bringing this issue to our attention, I tried face-landmarks-detection MediaPipe demo and it seems like there are total 03 missing keypoints so our relavant team will look into this issue and I believe they'll fix this issue soon
In the meantime, we welcome contributions from the community. If you have a solution to address these missing keypoints, please feel free to submit a pull request (PR). Our team will thoroughly review your PR and take appropriate action, including merging it if it meets our guidelines.
Thank you for your cooperation and patience.
System information
Describe the current behavior
keypoint
291
(a corner of the lips, see mesh_map) is not present inMEDIAPIPE_FACE_MESH_KEYPOINTS_BY_CONTOUR.lips
Describe the expected behavior
keypoint
291
should be included inMEDIAPIPE_FACE_MESH_KEYPOINTS_BY_CONTOUR.lips
Standalone code to reproduce the issue Lips corner not rendered correctly in face-landmarks-detection MediaPipe demo
Other info / logs I suspect the issue comes from constants.ts. See snippets below:
In particular,
connections.map(connection => connection[0]);
eliminateskeypoint
291
from the indices. This happens because291
appears as the second element in both[375, 291]
and[409, 291]
whereas61
(the other corner of the lips) appears as the first element in both[61, 146]
and[61, 185]
.Hope this helps!