IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.6k stars 4.83k forks source link

librealsense fails to claim USB interface on MacOS Monterey #9916

Open dominiccarrano opened 3 years ago

dominiccarrano commented 3 years ago

Required Info
Camera Model D415
Firmware Version N/A
Operating System & Version MacOS Monterey (12.0.1)
SDK Version 2.49
Language python

Issue Description

After upgrading my machine from macOS Catalina (10.15.6) to Monterey (12.0.1), all RealSense applications I have and pyrealsense2 code I’ve written -- which all previously ran without issue -- are broken. They all give me the same error message: failed to claim usb interface: 0, error: RS2_USB_STATUS_ACCESS. I’ve verified both physically and via System Preferences that the D415 is connected to the machine with a USB 3 connection using the same cable that the D415 was shipped with.

When I run rs-enumerate-devices, I get:

> rs-enumerate-devices
Could not create device - failed to set power state . Check SDK logs for details
 02/11 16:49:13,578 ERROR [0x70000cff3000] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_ACCESS

When I run realsense-viewer, I get:

> realsense-viewer
 02/11 16:52:08,669 ERROR [0x70000c115000] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_ACCESS
 02/11 16:52:08,669 ERROR [0x70000bf8c000] (sensor.cpp:537) acquire_power failed: failed to set power state
 02/11 16:52:08,673 WARNING [0x70000bf8c000] (rs.cpp:308) null pointer passed for argument “device”
 02/11 16:52:08,673 WARNING [0x70000bf8c000] (rs.cpp:2621) Couldn’t refresh devices - failed to set power state

I wondered if something was wrong with the device firmware and tried updating it, which gives me:

> rs-fw-update -l
connected devices:
 03/11 14:56:46,495 ERROR [0x70000b0b1000] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_ACCESS
RealSense error calling rs2_create_device(info_list:0x600003554e70, index:0):
    failed to set power state
 03/11 14:56:46,497 ERROR [0x119b34600] (sensor.cpp:537) acquire_power failed: failed to set power state

I uninstalled and then rebuilt librealsense from scratch for Monterey using the newest version of Xcode that’s Monterey compatible, but these issues still persist even after that. The librealsense homebrew page indicates that Monterey is supported. I’d greatly appreciate any guidance on fixing the failed to claim usb interface: 0, error: RS2_USB_STATUS_ACCESS problem for Monterey, thank you!

sam598 commented 2 years ago

i managed to build 2.50 with an m1 mac under Monterey 12.2.

I can run the realsense-viewer from the terminal with sudo, once i add the app manually to the accessibility tab in system preferences/security.

however it is unstable and i get the following errors: errors

RS: D435 FW: 05.12.12.100

This seems to confirm my fears and explains a lot of the behavior that has been seen so far. Apple has continued to add additional security features around camera access and privacy on both iOS and MacOS. My guess is that the Realsense SDK itself needs to request camera permissions before starting the device as an incoming video stream.

This would explain why Realsense cameras can be accessed as a webcam through first party apps. This would explain why the issue occurs on the newer OS with new security permissions, and not just newer hardware.

@MartyG-RealSense any update on this? These new features in MacOS are not going to go away, and it seems clear that this is an issue that Intel needs to provide a solution for. If they do not have the developers or resources to continue to support MacOS, then they need to officially delist it as a supported platform.

MartyG-RealSense commented 2 years ago

@sam598 I consulted with my colleagues further about the Monterey issue and was advised that the Supported Platforms information for MacOS only affirms official support up to the High Sierra OS. The subsequent releases after High Sierra were Mojave, Catalina, Big Sur and Monterey. Whilst Catalina and Big Sur work with the RealSense SDK, that compatibility is not officially guaranteed.

image

sam598 commented 2 years ago

@MartyG-RealSense that's even more frustrating as it is not possible to get MacOS devices with OSX High Sierra anymore.

If Intel no longer has the resources to investigate the privacy issue it would be better to clarify that Intel does not support MacOS at all.

It would also be good to know if this has officially been abandoned so the community can begin trying to work on a solution, as opposed to waiting on Intel. Especially since this issue has been around since at least November of last year.

MartyG-RealSense commented 2 years ago

I had a further discussion with my colleagues about MacOS support. As mentioned above, official support is guaranteed for MacOS High Sierra only. Intel have no plans to provide official support for MacOS versions newer than High Sierra.

However, Intel will accept contributions submitted by RealSense users (such as code pulls or installation instruction guides) that improve compatibility with MacOS versions newer than High Sierra.

sam598 commented 2 years ago

Hi everyone, after discussing the latest reports with Intel, the decision was made to file an official Intel bug report to investigate the issue further. I have added an Enhancement tag to this case to indicate that it should be kept open until the investigation process is completed.

@MartyG-RealSense since the investigation is over is there any insight that was found that could help the community solve this issue?

MartyG-RealSense commented 2 years ago

@sam598 There is no further information to add. At present the best theories in this discussion are your own one about camera permissions and the progress made by @hajdebliem earlier at https://github.com/IntelRealSense/librealsense/issues/9916#issuecomment-1026756545

Vangos commented 2 years ago

@MartyG-RealSense @hajdebliem Upon doing some further debugging, I strongly doubt it's a permissions issue.

After that, I tried to debug the RealSense code a little further. I started by simply executing the following program:


int main()
{
    rs2::context context;
    std::vector<rs2::device> devices = context.query_devices();
    int count = devices.size();

    printf("Connected devices: %d", count);

    return 0;
}

Really simple, huh? The file rs_context.hpp throws an exception in method query_devices():

So, I decided to go rogue and remove the error::handle(e); method call from the operator[] overload in file rs_device.hpp. Turns out now my code correctly returns the number of connected devices! Tried with 1 and 2 devices and worked fine!

However, even though it counts the connected devices, the rs2::device elements of the array are nullptrs.

@MartyG-RealSense Does this information help? Would you like to forward it to your team? I really think it's something straightforward to fix if Intel engineers are still committed to this project.

MartyG-RealSense commented 2 years ago

Hi @Vangos Thanks so much for your testing and information sharing! I will highlight your findings.

MartyG-RealSense commented 2 years ago

@Vangos In line with Intel's policy mentioned above in https://github.com/IntelRealSense/librealsense/issues/9916#issuecomment-1049042568 about RealSense users being encouraged to develop and submit changes to the Mac wrapper through a Pull Request (PR), would be possible to submit your change as a PR yourself, please?

https://github.com/IntelRealSense/librealsense/pulls

destr0ya commented 2 years ago

@MartyG-RealSense +1 experiencing same issue on Monterey 12.2 with Intel core. I am a customer of @Vangos and it's fundamental to our project to test with RealSense D415 on Mac. Would be great to get this sorted soon!

MartyG-RealSense commented 2 years ago

@destr0ya RealSense cameras should work on an Intel-based Mac with MacOS Catalina and Big Sur. At present though, manually editing the RealSense SDK file rs_device.hpp using the method described by @Vangos at https://github.com/IntelRealSense/librealsense/issues/9916#issuecomment-1054371868 may be the best solution available for using the cameras with Monterey.

ShiJie922 commented 2 years ago

I reinstalled the SDK via brew, and accidentally found that i can use ‘sudo’ to open ‘realsense-viewer’, but once I try this on other examples, 'Segmentation fault after sudo commands' occurs. Furthermore, By 'run Disk Utility, select the volume in the sidebar, then the First Aid tab, and click Repair Disk Permissions', after few minutes, most of the examples are work, except the IMU module on D435i. my english is not good, i hope my feedback helps.

MartyG-RealSense commented 2 years ago

Thanks so much @ShiJieNB for sharing your technique. Your English is very good. :)

Vangos commented 2 years ago

That's a bit misleading. It will, indeed, solve permission issues. But: the "Repair disk permissions" command does not exist on macOS Monterey, which means you are probably using an older version of macOS (please, correct me if I'm wrong).

Feel free to run a "First Aid," but it won't solve the issue, either.

Even if you give full disk access to your RealSense application, the problem persists.

It doesn't seem like a permissions issue, but rather a power issue (failed to set power state).

Regarding Brew: brew install librealsense does not support Apple Silicon, which includes all Apple computers released from 2020 onwards.

ShiJie922 commented 2 years ago

That's a bit misleading. It will, indeed, solve permission issues. But: the "Repair disk permissions" command does not exist on macOS Monterey, which means you are probably using an older version of macOS (please, correct me if I'm wrong).

Feel free to run a "First Aid," but it won't solve the issue, either.

Even if you give full disk access to your RealSense application, the problem persists.

It doesn't seem like a permissions issue, but rather a power issue (failed to set power state).

Regarding Brew: brew install librealsense does not support Apple Silicon, which includes all Apple computers released from 2020 onwards.

You're right, it was my fault, I forgot to clarify. The above method worked really well on my Monterey system at first, but the next day it didn't work again. I don't understand why, and I don't want to waste my time on mac os. Therefore, I made an Ubuntu dual system, and I have moved all the projects to the Ubuntu dual system, it worked well.

Vangos commented 2 years ago

Yeap, Linux works like a charm :-)

animaonline commented 2 years ago

Any plans to fix this?

destr0ya commented 2 years ago

Still an issue with Monterey 12.3

MartyG-RealSense commented 2 years ago

Hi @animaonline and @destr0ya Currently, the solution suggested earlier in this discussion by @Vangos at https://github.com/IntelRealSense/librealsense/issues/9916#issuecomment-1054371868 is the best known workaround available.

An official Intel bug report regarding MacOS Monterey issues has been filed but I have no information to provide about its progress at the time of writing this.

animaonline commented 2 years ago

@MartyG-RealSense thanks for the quick response, I just wanted to let you know that running the samples and realsense-viewer using sudo seems to work, so this is clearly not a macOS device problem.

MartyG-RealSense commented 2 years ago

Thanks so much @animaonline for the report about your own experiences with Monterey!

Dionysus326 commented 2 years ago

Hi I am using Monterey 12.3 with m1 and follow @Vangos approach, the problem still exists. I also tried @animaonline method running xcode in sudo and running examples, some passed, some failed. At the failed samples like realsense-viewer, still got the error ERROR [0x17085f000] (sensor.cpp:572) acquire_power failed: failed to set power state Anyone has the same issue?

animaonline commented 2 years ago

Hi I am using Monterey 12.3 with m1 and follow @Vangos approach, the problem still exists. I also tried @animaonline method running xcode in sudo and running examples, some passed, some failed. At the failed samples like realsense-viewer, still got the error ERROR [0x17085f000] (sensor.cpp:572) acquire_power failed: failed to set power state Anyone has the same issue?

I was running the precompiled sample binaries from the sdk folder. I haven’t tried running the xcode using sudo just yet.

Dionysus326 commented 2 years ago

@animaonline Thanks for the information. I will try it the other day. If I success I would update the info

Dionysus326 commented 2 years ago

@animaonline Could I ask you where you found the binaries. I could not find the binaries for mac...Got a feeling I am missing something.

animaonline commented 2 years ago

@Dionysus326 Hey, you can get the path of a program by running the 'where' command, e.g. where realsense-viewer

Dionysus326 commented 2 years ago

@Dionysus326 Hey, you can get the path of a program by running the 'where' command, e.g. where realsense-viewer How do you install realsense-view to your mac? Did you use brew install?

animaonline commented 2 years ago

@Dionysus326 I have done both, on two different macs. (Brew and sdk installation according to the docs)

Dionysus326 commented 2 years ago

@animaonline Thank you. I also installed on other mac without m1 chips using brew. Did not encounter any problems. And failed to install whether I am using brew or sdk on the mac using m1. At this point I just gave up

Vangos commented 2 years ago

OK, it works on Monterey 😄 Here's proof:

image

@MartyG-RealSense We have released this as part of our skeleton-tracking SDK. RealSense is a first-class citizen and we support it on Windows, Linux, and Mac.

Building RealSense binaries in a way that supports all macOS versions and chipsets is not trivial and requires re-compilation of additional C++ libraries. If anyone's interested, I can post a detailed tutorial. Just let me know.

MartyG-RealSense commented 2 years ago

Thanks so much @Vangos :) Thanks too for your past C# coordinate mapping tutorial, which is an excellent RealSense C# programming reference.

https://lightbuzz.com/intel-realsense-coordinate-mapping/

A detailed tutorial for setting up the RealSense SDK for MacOS Monterey would also be an excellent reference for Monterey users if you are able to publish one.

Your dedicated support of RealSense is very much appreciated!

cansik commented 2 years ago

@Vangos It would be great if you could give us some hints how to build it on an M1. I would like to include it into my pyrealsense2-macosx python repository to share pre-built M1 binaries too.

Vangos commented 2 years ago

I'm glad there's demand for this. I'll publish a detailed guide within the next few days :-)

Dionysus326 commented 2 years ago

@Vangos You are so nice! Thanks in advance! This would really help my team a lot.

otri commented 2 years ago

@Vangos exciting to hear you figured out the build process. That seems major. I guess you had to rebuild some support C++ libraries? Some summary pointers would be truly helpful as we are actively working on this issue right now.

Vangos commented 2 years ago

The tutorial is ready :-) I'm describing every step of the process.

https://lightbuzz.com/realsense-macos/

If you like it, please share to help fellow RealSense developers.

cansik commented 2 years ago

@Vangos Wow, great tutorial and the pre-compiled binary helps for the python wheel package too. Thank you very much, and let's hope Intel has not given up on RealSense for MacOS completely.

MartyG-RealSense commented 2 years ago

Yes, thanks so much again @Vangos for your excellent help!

cansik commented 2 years ago

First python build for version 3.6, 3.7, 3.8, 3.9, 3.10 is done. You can download the pre-built wheel packages here: https://github.com/cansik/pyrealsense2-macosx/releases/tag/macosx-v2.50.0

I can only test it on my own M1 Macbook, so would be great if someone could give it a try. Please be aware to run your python script with sudo, otherwise RealSense tells you that no device is connected.

ivanou-msi commented 2 years ago

The tutorial is ready :-) I'm describing every step of the process.

https://lightbuzz.com/realsense-macos/

If you like it, please share to help fellow RealSense developers.

I've gotten to this part:

"Just like we did for libusb earlier, go to ProductScheme and select librealsense from the drop-down list."

But there is no librealsense in the Scheme list. Is this a typo? Thanks!

Vangos commented 2 years ago

@ivanou-msi

But there is no librealsense in the Scheme list. Is this a typo? Thanks!

It's realsense2, like the screenshot. Fixed the typo, thanks for noticing!

ivanou-msi commented 2 years ago

@ivanou-msi

But there is no librealsense in the Scheme list. Is this a typo? Thanks!

It's realsense2, like the screenshot. Fixed the typo, thanks for noticing!

Ok, that's what I tried. But I don't see any new "RealSense" target after building. Any idea what I might be doing wrong?

cansik commented 2 years ago

I have updated the prebuilt python package pyrealsense2-macosx to version 2.50.0 with support for Intel and Apple Silicon. If you are interested in the fully automated build script to create pyrealsense2 & python binaries on MacOS BigSur and MacOS Monterey, have a look here: librealsense-python-mac.ps1

The prebuilt packages can be directly installed from PyPI:

pip install pyrealsense2-macosx

Thanks again to @Vangos for his great tutorial that helped me make this happen.

timprepscius commented 2 years ago

For anyone not using whatever things people are doing above this comment.

run: brew uninstall --ignore-dependencies libusb brew install libusb --HEAD

[the --HEAD might even be optional, but this is what I tried]

recompile thing librealsense and your own program

run the program which uses the camera with sudo

seems-to-work

MartyG-RealSense commented 2 years ago

Hi everyone, I just wanted to highlight that the brew install librealsense method of installing librealsense on Mac now lists support for MacOS Monterey and for Apple Silicon M1.

https://formulae.brew.sh/formula/librealsense

I also highly recommend the excellent in-depth installation guide published a little earlier in this discussion by @ivanou-msi at https://github.com/IntelRealSense/librealsense/issues/9916#issuecomment-1097268250

Vangos commented 2 years ago

https://formulae.brew.sh/formula/librealsense

@MartyG-RealSense This is great to hear. Keep in mind that you still need to uninstall and re-install libusb via Brew. Moreover, RealSense apps (including the RealSense Viewer) must run using sudo.

One more thing: Brew builds are not universal, so if you still need to support Apple Silicon and Intel chips with the same binaries, you should follow our installation guide and compile from scratch.

MartyG-RealSense commented 2 years ago

Hi everyone,

As there is now a clear and detailed installation guide for MacOS Monterey on Intel and M1 Macs, is it okay if this case is now closed as resolved? If any of you have an opinion that it should be kept open, I welcome your input. Thanks!

dominiccarrano commented 2 years ago

I think we can close this thread after the next major macOS release is available, but that we should leave it open for now to let the community discuss and help troubleshoot Monterey installation issues since that's the latest version of macOS.

MartyG-RealSense commented 2 years ago

MacOS 13 could be several months away yet at the time of writing this.

blackccpie commented 2 years ago

Hi everyone, does anyone known the true reason why sudo execution is now required on MacOS 12? Is this something that we can work on and expect to correct for the future? Having to launch apps using the terminal is kind of a showstopper for a standard app user. Thanks!