Face mesh aspect is distorted when drawn over preview view (width is too narrow)

[MonteCreasor]

Your code does not correctly map the face mesh points from the analysis aspect ratio to the preview aspect ratio for both the rear and front cameras. This new adjustPoint() method below will properly handle both front and rear cameras and in both cases, the mesh will be the exact correct size and perfectly aligned over the user's face. I've also included a new adjustSize that will ensure that the bounding box is scaled properly for both cameras and is aligned over the mesh for the front camera. This is just a quick fix. In my version, I will be optimizing this code.

fun adjustPoint(
    point: PointF,
    imageWidth: Int,
    imageHeight: Int,
    screenWidth: Int,
    screenHeight: Int,
    isFrontCamera: Boolean = true
): PointF {
    val imageAspectRatio = imageWidth.toFloat() / imageHeight
    val screenAspectRatio = screenWidth.toFloat() / screenHeight

    val scaleFactor = if (imageAspectRatio > screenAspectRatio) {
        // Width is the limiting factor.
        screenHeight.toFloat() / imageHeight
    } else {
        // Height is the limiting factor.
        screenWidth.toFloat() / imageWidth
    }

    // Calculate the horizontal offset to center the mesh.
    val offsetX = (screenWidth - (imageWidth * scaleFactor)) / 2

    // Adjust x-coordinate for mirroring if using the front camera.
    val x = if (isFrontCamera) {
        screenWidth - ((point.x * scaleFactor) + offsetX) // Mirror the x-coordinate
    } else {
        (point.x * scaleFactor) + offsetX
    }

    val y = point.y * scaleFactor // No change in the y scaling

    return PointF(x, y)
}

fun adjustSize(
    size: Size,
    imageWidth: Int,
    imageHeight: Int,
    screenWidth: Int,
    screenHeight: Int,
    isFrontCamera: Boolean = true
): Size {
    val imageAspectRatio = imageWidth.toFloat() / imageHeight
    val screenAspectRatio = screenWidth.toFloat() / screenHeight

    val scaleFactor = if (imageAspectRatio > screenAspectRatio) {
        // Width is the limiting factor.
        screenHeight.toFloat() / imageHeight
    } else {
        // Height is the limiting factor.
        screenWidth.toFloat() / imageWidth
    }

    // Adjust x-coordinate for mirroring if using the front camera.
    val width = if (isFrontCamera) {
        size.width * -scaleFactor/// imageWidth * screenWidth
    } else {
        size.width * scaleFactor/// imageWidth * screenWidth
    }

    val height = size.height * scaleFactor/// imageHeight * screenHeight
    return Size(width, height)
}

[chouaibMo]

Hi @MonteCreasor, Thank you for bringing this to my attention. Your proposed changes sound promising and would greatly improve the functionality. Could you please submit a Pull Request with these adjustments so that we can review and merge them into the main branch? Your contribution is much appreciated!

[MonteCreasor]

I'll do that once I have a better solution. I'm making my camera app work so that the face tracking will show the overlay at the correct location no matter how you rotate the phone. Your version and even with the fix I provided will not do that. It's complicated to get that working.

On Tue, Jan 16, 2024, 4:49 PM Chouaib Mounaime @.***> wrote:

Hi @MonteCreasor https://github.com/MonteCreasor, Thank you for bringing this to my attention. Your proposed changes sound promising and would greatly improve the functionality. Could you please submit a Pull Request with these adjustments so that we can review and merge them into the main branch? Your contribution is much appreciated!

— Reply to this email directly, view it on GitHub https://github.com/chouaibMo/MLKit-Jetpack-Compose/issues/1#issuecomment-1894569879, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJCBTSTBJHVOB7DH2AYBR3YO3YWBAVCNFSM6AAAAABBWJQPQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGU3DSOBXHE . You are receiving this because you were mentioned.Message ID: @.***>