PoseLandmarker and GestureRecognizer can take 30+ seconds to load in the first time.

[galharth]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Ios 17.0.3

MediaPipe Tasks SDK version

0.10.10

Task name (e.g. Image classification, Gesture recognition etc.)

Gesture recognition and poseLandmarker

Programming Language and version (e.g. C++, Python, Java)

Javascript

Describe the actual behavior

PoseLandmarker and GestureRecognizer can take 30+ seconds to load in the first time.

Describe the expected behaviour

PoseLandmarker and GestureRecognizer loading will be faster and there will be a progress bar for the user.

Standalone code/steps you may have used to try to get what you need

This code can take 30+ seconds to load on iOS webview. 

import { PoseLandmarker, FaceLandmarker, Holistic, GestureRecognizer, FilesetResolver, DrawingUtils } from "./tasks-vision-0-10-10.js";

FilesetResolver.forVisionTasks(
            "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.10/wasm"
        ).then((vision) => {

            // Create promises for loading Pose Landmarker and Gesture Recognizer
            const poseLandmarkerPromise = PoseLandmarker.createFromOptions(vision, {
                baseOptions: {
                    modelAssetPath: `https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_lite/float16/1/pose_landmarker_lite.task`,
                    delegate: "CPU"
                },
                canvas: document.getElementById('canvas-test2'),
                runningMode: "VIDEO",
                numPoses: 1
            });

            const gestureRecognizerPromise = GestureRecognizer.createFromOptions(vision, {
                baseOptions: {
                    modelAssetPath: "https://storage.googleapis.com/mediapipe-tasks/gesture_recognizer/gesture_recognizer.task",
                    delegate: "CPU",
                },
                canvas: document.getElementById('canvas-test'),
                runningMode: "VIDEO",
                numHands: 2
            });

            // Use Promise.all to load both in parallel
            Promise.all([poseLandmarker Promise, gestureRecognizerPromise])
                .then(([poseLandmarker, gestureRecognizer]) => {

});});

Other info / Complete Logs

No response

[ayushgdev]

Hello @galharth Please allow us sometime to investigate this issue. If it could be consistently recreated at our end, we shall discuss this internally how we can improve model loading time. However, just a heads up, model optimizations without accuracy deteriorations can take time. Meanwhile, if progress bar can be implemented, we will dig into the feature implementation. For now, we shall classify it as performance issue.

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[galharth]

any update?

[galharth]

@ayushgdev Hey, is there a workaround maybe to improve the speed?