Macos Catalina Hand tracking

[nov08demo]

Hi, I am trying to run the hand tracking example on my macbook pro 16 which is running catalina. I followed the installation process successfully and ran the following commands:

$ bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 \ mediapipe/examples/desktop/multi_hand_tracking:multi_hand_tracking_cpu $ GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/multi_hand_tracking/multi_hand_tracking_cpu \ --calculator_graph_config_file=mediapipe/graphs/hand_tracking/multi_hand_tracking_desktop_live.pbtxt

and got the following output:

I0225 10:35:16.197036 499596736 demo_run_graph_main.cc:53] Initialize the calculator graph. I0225 10:35:16.200453 499596736 demo_run_graph_main.cc:57] Initialize the camera or load the video. I0225 10:35:18.714452 499596736 demo_run_graph_main.cc:78] Start running the calculator graph. I0225 10:35:18.714884 499596736 demo_run_graph_main.cc:83] Start grabbing and processing frames. I0225 10:35:19.715039 499596736 demo_run_graph_main.cc:135] Shutting down. I0225 10:35:19.954938 499596736 demo_run_graph_main.cc:149] Success!

it runs but then suddenly shuts down with no warnings or errors. The camera is working and I allowed the program permission to access the camera when the initial permissions window popped up. I also attempted running the same example on a separate machine runningMacOs Mojave, which gave me no issues and ran perfectly. Any reason why it doesnt run on Catalina or mac pro 16"?

[mcclanahoochie]

Related https://github.com/google/mediapipe/issues/424

Does providing a video work?
--input_video_path=/tmp/video.mp4 Our team hasn't had the opportunity to test on the new Mac OS yet.

Have you tried this ?

[nov08demo]

Hi I have tried the permissions in settings, still no luck

[Akathian]

Related #424

Does providing a video work? --input_video_path=/tmp/video.mp4 Our team hasn't had the opportunity to test on the new Mac OS yet.

Have you tried this ?

Having the same issue here. This did not work for me either.

[nov08demo]

Hi have you guys had a chance to test on Catalina yet? We are still encountering the same issue

[mcclanahoochie]

Update: I have just [in-place] upgraded my 2017 macbook 15" to macOS Catalina (10.15.3), and had no issues running the multi-hand desktop demo using the built-in webcam.

Granted, this demo was running fine before the upgrade, but at least I can confirm the upgrade to catalina didn't break things (on my machine). Also, I don't remember doing any special permissions (or was never prompted to by the OS).

The unknowns are if there is something unique about the 16" macbook, and/or something odd about a "fresh/clean" install of Catalina OS (what the 16" comes with), or just something else not setup correctly.

Do all examples have this problem for people with 16" macbooks? What version of OpenCV are people using? Does the PhotoBooth app work fine?

[mcclanahoochie]

list of things to try:

• Terminal /iTerm Camera permissions
• try video via: --input_video_path=/tmp/video.mp4
• see if a simple separate python or c++ example works
• try a command line capture program
• going through other suggestions here
• try to run "tcutil reset Camera" in a terminal (no quotes)

if i think of others i can update the list (or if something starts working let me know)

[nov08demo]

Hi, we seem to have it working now an older mac on catalina. The issue seems to persist on the new 16" macs tho, tried the different suggestion you made.

[mcclanahoochie]

Possible solution from here: https://stackoverflow.com/a/58997222 (slightly modified by me below)

Info: Every app on Mac OS/iOS/watches OS needs privacy permission from the user before accessing features like camera/ mic/ gallery etc. These permissions are stored in a property list file generally "info.plist". Now as this app will run on the mac itself you need to specify permissions before running.

Solution: Go to the following on your mac (i'm using iTerm2 to run MediaPipe): Finder -> Applications -> iTerm (right-click to show package contents) -> Contents -> Info.plist

Now open "Info.plist" file with any plain text editor and paste the following code right after the opening tag, which is probably in 4th or 5th line:

<key>NSCameraUsageDescription</key>
<string>An application in iTerm2 wants to use the camera.</string>

(the above <> was taken from my working iTerm.app info.plist)

Again, my system is working, so don't know for sure if it work, but seems to for some others (original post)

[agrawalramakant]

Still does not work for me in Mac 16 inch 2019 model.

[lolanchen]

Having the same issue when trying to run the face_mesh example, also on the 16' Macbook pro. video input does work tho, it just won't run on the webcam.

[peterjzhou]

Same problem met here for 16' MacBook Pro.

[lcwill]

I've encountered the same issue described in the original post, with a 16" 2019 MBP running Catalina (10.15.6).

The issue for me seems to be in this section of the code, at the beginning of the image frame processing loop:

    // Capture opencv camera or video frame.
    cv::Mat camera_frame_raw;
    capture >> camera_frame_raw;
    if (camera_frame_raw.empty()) break;  // End of video.

Apparently, the first frame of the video is empty, i.e. camera_frame_raw.empty() returns true, perhaps due to there being some delay between the camera being initialized and video being captured. Modifying the above code as follows allowed me to workaround this issue:

    // Capture opencv camera or video frame.
    cv::Mat camera_frame_raw;
    capture >> camera_frame_raw;
    if (camera_frame_raw.empty()) {
      if (!load_video) {
        // Ignore empty frames from camera feed
        continue;
      }
      LOG(INFO) << "Empty frame, end of video reached";
      break;
    }

[yasithdev]

This solution worked for me!

[abreu4]

Also worked for me

[ianni67]

I got the same problem and fixed it the same way on my ubuntu box. Thank you for pointing it out. This solution should go upstream, as the issue is well-known also for linux. All programs acquiring live video from webcam with OpenCV should have this check, independently of the OS.

[S3od22]

[]()

[mcclanahoochie]

Thanks for the updates. The fix has been integrated and should be in the next release.

[ianni67]

Sorry for bothering again. I would suggest to extend the fix to the python version of the demo software, too. In my experience the python version of OpenCV suffers from the same issue.