Javscript Mediapipe Hands Solution not working with USB desktop camera

jose-b1253 commented 2 years ago

System information (Please provide as much relevant information as possible) Camera: Depstech Desktop USB Camera

Used mediapipe template code and modified it for Typescript.
OS Platform and Distribution: Windows 11
Browser and version if the issue happens on browser: Firefox v99.0 (64-bit)/ Edge 100.0.1185.29 (64-bit)/ Chrome
Programming Language and version: Javascript/Typescript
MediaPipe version: v0.8.9
Solution ( e.g. FaceMesh, Pose, Holistic ): Hands

Describe the current behavior: Mediapipe Hands solution displays the first frame from the video input, but freezes afterwards indefinitely.

Describe the expected behavior: Mediapipe Hands solution analyzes each frame from the video input and draws landmarks.

Standalone code to reproduce the issue: Provide a reproducible test case that is the bare minimum necessary to replicate the problem. If possible, please share a link to Colab/repo link /any notebook:

https://google.github.io/mediapipe/solutions/hands.html#javascript-solution-api We used the template code to be sure, and doesn't work either.

Other info / Complete Logs : Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached

To clarify, the USB camera being used is fully functional and also observed the same behavior with a second USB camera. We suspect that the problem occurs when the original video input is sent to the hands solution.

i.e

await hands.send({image: videoElement});

This is the logs from inspect element when the first frame loads.

I0000 00:00:1649258560.000000       1 gl_context_webgl.cc:151] Successfully created a WebGL context with major version 3 and handle 3 [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
I0000 00:00:1649258560.000000       1 gl_context.cc:351] GL version: 3.0 (OpenGL ES 3.0 (WebGL 2.0)) [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
W0000 00:00:1649258560.000000       1 gl_context.cc:841] OpenGL error checking is disabled [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
WebGL warning: drawArraysInstanced: Tex image TEXTURE_2D level 0 is incurring lazy initialization.
E0000 00:00:1649258563.400000       1 calculator_graph.cc:809] INVALID_ARGUMENT: CalculatorGraph::Run() failed in Run: [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
Packet timestamp mismatch on a calculator receiving from stream "input_frames_gpu". Current minimum expected timestamp is 4900001 but received 4900000. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment. [type.googleapis.com/mediapipe.StatusList='\n\xea\x02\x08\x03\x12\xe5\x02Packet timestamp mismatch on a calculator receiving from stream \"input_frames_gpu\". Current minimum expected timestamp is 4900001 but received 4900000. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment.'] [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
=== Source Location Trace: === [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
third_party/mediapipe/framework/tool/status_util.cc:139 [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
<empty string> [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
F0000 00:00:1649258563.400000       1 solutions_wasm.embind.cc:679] Check failed: absl::OkStatus() == ::util::status_internal::AsStatus((graph_->WaitUntilIdle())) (OK vs. INVALID_ARGUMENT: CalculatorGraph::Run() failed in Run: [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
Packet timestamp mismatch on a calculator receiving from stream "input_frames_gpu". Current minimum expected timestamp is 4900001 but received 4900000. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment. [type.googleapis.com/mediapipe.StatusList='\n\xea\x02\x08\x03\x12\xe5\x02Packet timestamp mismatch on a calculator receiving from stream \"input_frames_gpu\". Current minimum expected timestamp is 4900001 but received 4900000. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment.'] [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
=== Source Location Trace: === [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
third_party/mediapipe/framework/tool/status_util.cc:139 [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
) [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
*** Check failure stack trace: *** [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)

Aborted(native code called abort()) [react_devtools_backend.js:3973:25](moz-extension://3e64e843-5eb3-44f5-bbe2-1ece4b95e3cf/build/react_devtools_backend.js)
Uncaught (in promise) RuntimeError: Aborted(native code called abort())
    abort hands_solution_simd_wasm_bin.js:9
    _abort hands_solution_simd_wasm_bin.js:9
    SolutionWasm$send hands_solution_simd_wasm_bin.js line 9 > Function:9
    node_modules bundle.js:11977
    G hands.js:14
    node_modules NextJS
    b hands.js:15
    promise callback*g hands.js:15
    promise callback*g hands.js:15
    node_modules bundle.js:9954
    ta hands.js:15
    I hands.js:16
    node_modules bundle.js:11861
    node_modules bundle.js:12290
    G hands.js:14
    node_modules NextJS
    node_modules bundle.js:9954
    ta hands.js:15
    I hands.js:16
    node_modules bundle.js:12289
    onFrame Camera.tsx:128
    Q camera_utils.js:22
    node_modules bundle.js:9369
    P camera_utils.js:22
    Q camera_utils.js:22
    node_modules bundle.js:9369
    P camera_utils.js:22
    node_modules bundle.js:9387
    promise callback*Q camera_utils.js:22
    node_modules bundle.js:9369
    P camera_utils.js:22
    node_modules bundle.js:9387
    promise callback*Q camera_utils.js:22
    node_modules bundle.js:9369
    P camera_utils.js:22
    node_modules bundle.js:9379
    O camera_utils.js:22
    node_modules bundle.js:9357
[hands_solution_simd_wasm_bin.js:9:25546](https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands_solution_simd_wasm_bin.js)

kuaashish commented 1 year ago

Hello @JoseBiescas, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions. Thank you

github-actions[bot] commented 1 year ago

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

github-actions[bot] commented 1 year ago

This issue was closed due to lack of activity after being marked stale for past 7 days.

google-ml-butler[bot] commented 1 year ago

Are you satisfied with the resolution of your issue? Yes No

harvester-klirk commented 4 months ago

still having the same problem in reactjs

google-ai-edge / mediapipe

Javscript Mediapipe Hands Solution not working with USB desktop camera #3241