Javascript: mediapipe/pose is not working inside webworker

[hsakbo]

System information (Please provide as much relevant information as possible)

Kernel: 5.19.8-arch1-1
Uptime: 1 hour, 57 mins
Packages: 1138 (pacman)
Shell: bash 5.1.16
Resolution: 1920x1080
DE: Xfce 4.16
WM: Xfwm4
WM Theme: Default
Theme: Adwaita-dark [GTK2/3]
Icons: Adwaita [GTK2/3]
Terminal: xfce4-terminal
Terminal Font: Monospace 12
CPU: Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
GPU: NVIDIA GeForce MX330 (Not used)
GPU: Intel CometLake-U GT2 [UHD Graphi
Memory: 7179MiB / 15825MiB

• Have I written custom code (as opposed to using a stock example script provided in MediaPipe): Yes

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4): ArchLinux

• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Not tested

• Browser and version (e.g. Google Chrome, Safari) if the issue happens on browser: Chromium Version 105.0.5195.102 (Official Build) Arch Linux (64-bit)

• Programming Language and version ( e.g. C++, Python, Java): Javascript

• MediaPipe version: 0.5.1635988162

• Bazel version (if compiling from source): Not compiled

• Solution ( e.g. FaceMesh, Pose, Holistic ): Pose

• Android Studio, NDK, SDK versions (if issue is related to building in Android environment): Not tested

• Xcode & Tulsi version (if issue is related to building for iOS): Not tested

Describe the current behavior: There are several issues related to @mediapipe/pose library. One issue is that the path resolved internally is wrong, meaning it doesn't follow the path set by locateFile but rather the address of the script itself under webworker.

Another issue is it is trying to execute importScripts to a tflite file and failing in the process.

Describe the expected behavior: I would expect a correct resolution of path, meaning to look for files set by locateFile in webworker context. Additionally, for this library to execute pose.send without issues

Standalone code to reproduce the issue: Provide a reproducible test case that is the bare minimum necessary to replicate the problem. If possible, please share a link to Colab/repo link /any notebook: https://github.com/hsakbo/mediapipe-pose-webworker-demo

Other info / Complete Logs : Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached

[kuaashish]

Hi @hsakbo, Please refer to this running thread #2506(Comment) for better understanding on your query. Meanwhile as a workaround you can follow the mentioned in this thread #2506 to run the MediaPipe models in the web worker. Thank you!

[hsakbo]

@kuaashish I have tested workarounds mentioned under #2506, they don't happen to work as expected. pose.js is calling case 2 in my instance, trying to load a tflite model with importScripts api, as demonstrated by screenshot above

[kuaashish]

Hello @hsakbo,

Our new MP Tasks can run in the web worker, Please go through new tasks api and try out the configuration, Which might fit in your use case. Thank you!!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.