Face Mesh Masks Not Loading in Firefox Despite Graphics Acceleration Being On

[anujSatschel]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Mac OS

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

FireFox

Programming Language and version

Javascript

MediaPipe version

No response

Bazel version

No response

Solution

FaceMesh

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Face Mesh Masks Not Loading in Firefox Despite Graphics Acceleration Being On

Describe the expected behaviour

Face Mesh mask is not loading in Firefox

Standalone code/steps you may have used to try to get what you need

https://discuss.ai.google.dev/t/face-mesh-masks-not-loading-in-firefox-despite-graphics-acceleration-being-on/42168?u=anuj_srivastav

Please refer to the URL, and attached screenshot of the error there

Other info / Complete Logs

No response

[kuaashish]

Hi @anujSatschel,

Could you please provide the complete steps you are following from our documentation, or share the steps that will help us understand the issue? If needed, we can reproduce the issue on our end.

Thank you!!

[anujSatschel]

Hi @kuaashish sure , Steps to reproduce:

1. Open Firefox
2. Enable WebGL (set webgl.force-enabled to true in about:config).
3. Open my web application that uses MediaPipe's FaceMesh feature.
4. Attempt to load the face mesh mask by enabling the camera.

Expected result: The face mesh mask should be visible on the canvas. Actual result: The face mesh mask does not load or display on the canvas.

Below is the part of the code where I handle the initialization of MediaPipe and webcam stream: it's just an example of my approach

const enableCam = useCallback(async (draw?: boolean) => {
    try {
        if (webcamRunning.current) {
            // Webcam is already running
            return;
        }

        const devices = await navigator.mediaDevices.enumerateDevices();
        let frontCamera = devices.find(device => 
            device.kind === 'videoinput' && 
            device.label.toLowerCase().includes('front')
        );

        if (!frontCamera) {
            frontCamera = devices.find(device => device.kind === 'videoinput');
        }

        const videoStream = {
            video: {
                deviceId: frontCamera?.deviceId,
                facingMode: 'user',
                width: { ideal: 1920 },
                height: { ideal: 1080 },
            },
        };

        const stream = await navigator.mediaDevices.getUserMedia(videoStream);
        videoElement.current.srcObject = stream;
        webcamRunning.current = true;

        const filesetResolver = await FilesetResolver.forVisionTasks(
            'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.3/wasm'
        );

        faceLandmarker.current = await FaceLandmarker.createFromOptions(filesetResolver, {
            baseOptions: {
                modelAssetPath: `https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task`,
                delegate: 'GPU',
            },
            outputFaceBlendshapes: true,
            runningMode: 'VIDEO',
            numFaces: 1,
        });

        if (draw) {
            videoElement.current.addEventListener('loadeddata', drawCanvas);
        }
    } catch (error) {
        console.error('Error accessing webcam:', error);
    }
}, []);

const drawCanvas = useCallback(async () => {
    const canvasCtx = canvasElement.current?.getContext('2d');
    const video = videoElement.current;

    if (!canvasCtx || !video) {
        return;
    }

    const startTimeMs = performance.now();
    if (lastVideoTime.current !== video.currentTime) {
        lastVideoTime.current = video.currentTime;
        results.current = faceLandmarker.current?.detectForVideo(video, startTimeMs) ?? {};
    }

    if (results.current?.faceLandmarks) {
        // Draw face landmarks on canvas
        results.current.faceLandmarks.forEach(landmarks => {
            drawingUtils.drawConnectors(
                landmarks,
                FaceLandmarker.FACE_LANDMARKS_TESSELATION,
                { color: '#C0C0C070', lineWidth: 1 }
            );
        });
    }

    window.requestAnimationFrame(drawCanvas);
}, []);

The face landmark detection works flawlessly in Chrome and Edge but fails to draw in Firefox(in some cases). Attached is the screenshot of the error

below