search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.43k stars 4.8k forks source link

point cloud generation- x and y coordinates not accurate #12503

Open swishswish123 opened 7 months ago

swishswish123 commented 7 months ago
Required Info
Camera Model D400
Firmware Version
Operating System & Version MacOS
Kernel Version (Linux Only)
Platform
SDK Version
Language Python
Segment

Issue Description

I am generating a point cloud of an object with known dimensions and known distance from the camera. When I check the coordinates, the z distance seems correct, but the dimensions in the x and y directions is always smaller than it should be.

This is the method I use as part of a class to generate the point cloud from the images:

    def generate_pc(self, save_file):
        pc = rs.pointcloud()
        # apply texture to depth pc
        pc.map_to(self.color_frame)
        pointcloud = pc.calculate(self.depth_frame)
        pointcloud.export_to_ply(save_file, self.color_frame)
        self.pointcloud = pointcloud

        return pointcloud

Would appreciate any help!

Thanks so much

MartyG-RealSense commented 6 months ago

It sounds as though the reset code attempted to reset the camera by disconnecting it and then re-connecting it but then was unable to find it when trying to re-connect after disconnection.

When the Viewer or your program is run after the computer is started up, does it always work the first time? There are some cases where a program will work the first time it is run but not the second time.

swishswish123 commented 6 months ago

Yes that makes sense as when I run the viewer it takes a few times before it works, giving me the power state error. The number of times I have to run it before it works is unpredictable. Do you know why this happens?

MartyG-RealSense commented 6 months ago

It sounds more like a hardware issue than a program issue. It could be that the brand of USB controller chosen by your computer's manufacturer to control the USB ports may not get along well with your RealSense camera.

A way to test this would be to plug the camera into a USB hub instead of directly into the computer's USB port if you are not using a hub already. A hub uses its own brand of USB controller for its ports instead of the one used by the computer's ports.

You could also test whether reliability improves if you unplug the camera from the port, wait a couple of seconds and then re-insert it. Doing so would have the same effect as resetting the camera.

swishswish123 commented 6 months ago

I can't seem to get it to work 🙁

I tried with a different USB hub than I was using (I was previously using the apple official one).

The unplugging the camera for a few seconds and plugging it back in seemed to work for the realsense-viewer for a bit but now it's gone back to being random...

MartyG-RealSense commented 6 months ago

https://github.com/IntelRealSense/librealsense/issues/11815 seems to be relevant to the issues that you are having regarding Monterey, the need for sudo and 'Failed to set power state' errors.

MartyG-RealSense commented 5 months ago

Hi @swishswish123 Was the information in https://github.com/IntelRealSense/librealsense/issues/11815 helpful to your problem, please? Thanks!

swishswish123 commented 5 months ago

Thanks @MartyG-RealSense, sorry for taking so long to reply, I've not yet had the chance to test it out but I will try to do it this week and get back to you

MartyG-RealSense commented 5 months ago

It's no problem at all. I look forward to your next report. Good luck!

HasanTheSyrian commented 5 months ago

I have the same issue. I still can't get the viewer to work on my Mac running Sonoma 14.2.1 (i havent tried pyrealsense-macos)

I'm on v2.54.2 with the latest firmware version I flashed using a Windows desktop. I'm using the brew install.

I can launch the viewer normally without sudo, but as soon as I plug in the camera it crashes.

If I launch the viewer with: no sudo, usb-c to usb-c cable:

 27/01 15:06:08,526 INFO [0x1d7075000] (context.cpp:336) Found 1 RealSense devices (mask 0xff)
 27/01 15:06:08,542 ERROR [0x16b73f000] (handle-libusb.h:127) failed to claim usb interface: 0, error: 
 RS2_USB_STATUS_ACCESS
 27/01 15:06:08,543 ERROR [0x16b59b000] (sensor.cpp:661) acquire_power failed: failed to set power state
 27/01 15:06:08,544 WARNING [0x16b59b000] (rs.cpp:312) null pointer passed for argument "device"
 27/01 15:06:08,544 WARNING [0x16b59b000] (rs.cpp:2700) Couldn't refresh devices - failed to set power state
 27/01 15:06:08,589 INFO [0x1d7075000] (rs.cpp:2697) Framebuffer size changed to 2418 x 1562
 27/01 15:06:08,589 INFO [0x1d7075000] (rs.cpp:2697) Scale Factor is now 1
 Assertion failed: (list_empty(&darwin_cached_devices)), function darwin_init, file darwin_usb.c, line 605.
 [1]    6273 abort      realsense-viewer

If I launch the viewer with: sudo, usb-c to usb-c cable:

 27/01 15:14:50,256 INFO [0x1d7075000] (context.cpp:336) Found 1 RealSense devices (mask 0xff)
 27/01 15:14:50,304 INFO [0x1d7075000] (rs.cpp:2697) Framebuffer size changed to 2418 x 1562
 27/01 15:14:50,304 INFO [0x1d7075000] (rs.cpp:2697) Scale Factor is now 1
[1]    6405 segmentation fault  sudo realsense-viewer

If I launch the viewer with: no sudo, usb-a to usb-c cable w/converter:

 27/01 15:17:43,386 INFO [0x1d7075000] (context.cpp:336) Found 1 RealSense devices (mask 0xff)
 27/01 15:17:43,405 ERROR [0x16e3af000] (handle-libusb.h:127) failed to claim usb interface: 0, error: RS2_USB_STATUS_ACCESS
 27/01 15:17:43,406 ERROR [0x16e20b000] (sensor.cpp:661) acquire_power failed: failed to set power state
 27/01 15:17:43,406 WARNING [0x16e20b000] (rs.cpp:312) null pointer passed for argument "device"
 27/01 15:17:43,407 WARNING [0x16e20b000] (rs.cpp:2700) Couldn't refresh devices - failed to set power state
 27/01 15:17:43,451 INFO [0x1d7075000] (rs.cpp:2697) Framebuffer size changed to 2418 x 1562
 27/01 15:17:43,451 INFO [0x1d7075000] (rs.cpp:2697) Scale Factor is now 1
Assertion failed: (list_empty(&darwin_cached_devices)), function darwin_init, file darwin_usb.c, line 605.
[1]    6450 abort      realsense-viewer

If I launch the viewer with: sudo, usb-a to usb-c cable w/converter:

 27/01 15:18:54,924 INFO [0x1d7075000] (context.cpp:336) Found 1 RealSense devices (mask 0xff)
 27/01 15:18:54,970 INFO [0x1d7075000] (rs.cpp:2697) Framebuffer size changed to 2418 x 1562
 27/01 15:18:54,970 INFO [0x1d7075000] (rs.cpp:2697) Scale Factor is now 1
[1]    6459 segmentation fault  sudo realsense-viewer

You were discussing the cable, but it seems like it doesn't have an effect.

MartyG-RealSense commented 5 months ago

@HasanTheSyrian There was a similar problem at https://github.com/IntelRealSense/librealsense/issues/12307#issuecomment-1812688156 with realsense-viewer when using sudo.

One RealSense Mac user suggested dealing with the Failed to set power state error by using brew install libusb first before doing brew install librealsense

HasanTheSyrian commented 5 months ago

Unfortunately did not work.

swishswish123 commented 5 months ago

I tried what jackjansen said in #11815 of trying different ports and at the moment it seems like the port furthest from the screen, on the side of the power supply works. I am using USB-C to USB-A cable, then the apple USB-A to USB-C adaptor and connect that to my Mac.

When I say works, I mean my python script AND the realsense-viewer seem to be running properly 🥳

So @HasanTheSyrian maybe you can try that way? You should also try inserting the USB-C in both ways, as to me it seems to make a difference which way I plug in the apple adaptor. (For the realsense viewer this is of course after doing brew install libsub then brew install librealsense)

Let's hope this doesn't stop working like last time! Thanks so much @MartyG-RealSense for the help :)

HasanTheSyrian commented 5 months ago

I tried what jackjansen said in #11815 of trying different ports and at the moment it seems like the port furthest from the screen, on the side of the power supply works. I am using USB-C to USB-A cable, then the apple USB-A to USB-C adaptor and connect that to my Mac.

When I say works, I mean my python script AND the realsense-viewer seem to be running properly 🥳

So @HasanTheSyrian maybe you can try that way? You should also try inserting the USB-C in both ways, as to me it seems to make a difference which way I plug in the apple adaptor. (For the realsense viewer this is of course after doing brew install libsub then brew install librealsense)

Let's hope this doesn't stop working like last time! Thanks so much @MartyG-RealSense for the help :)

I ordered a UBS-C - USB-A adapter, I'll report back when it comes.

MartyG-RealSense commented 5 months ago

Thanks so much @swishswish123 for your advice to @HasanTheSyrian :)

HasanTheSyrian commented 5 months ago

@MartyG-RealSense By the way, if I install Ubuntu [1,2] on my Mac (yes there is a native M1 Linux kernel with reverse-engineered drivers and everything) would it work considering it's ARM? This could be a solution for others as well since a lot of us (I think :3) also like Linux.

MartyG-RealSense commented 5 months ago

I cannot recall a past case in which Ubuntu has been used with RealSense on a Mac. It was more usual for Windows to be used via MacOS' Boot Camp Assistant utility. Boot Camp does not work with Apple Silicon Macs though, but my understanding is that it is possible to install multiple OS on M1 / M2 Macs and access them via the Start-Up options by holding down the power button.

librealsense can be compiled on Ubuntu on Arm devices (such as Raspberry Pi, Nvidia Jetson and other brands of Arm-based single board computers). I do not have an example of its installation and use on Ubuntu on Mac though.

Looking at the details of Asahi, it seems like a very good match for M1 and M2 Macs. librealsense can be built from source code on non-Ubuntu flavors of Linux if compiled in RSUSB = true mode, as RSUSB is not dependent on Linux versions or kernel versions and does not need to have a patch applied to the kernel.

HasanTheSyrian commented 5 months ago

I cannot recall a past case in which Ubuntu has been used with RealSense on a Mac. It was more usual for Windows to be used via MacOS' Boot Camp Assistant utility. Boot Camp does not work with Apple Silicon Macs though, but my understanding is that it is possible to install multiple OS on M1 / M2 Macs and access them via the Start-Up options by holding down the power button.

librealsense can be compiled on Ubuntu on Arm devices (such as Raspberry Pi, Nvidia Jetson and other brands of Arm-based single board computers). I do not have an example of its installation and use on Ubuntu on Mac though.

Looking at the details of Asahi, it seems like a very good match for M1 and M2 Macs. librealsense can be built from source code on non-Ubuntu flavors of Linux if compiled in RSUSB = true mode, as RSUSB is not dependent on Linux versions or kernel versions and does not need to have a patch applied to the kernel.

The USB dongle didn't work... but I installed Asahi Linux and...

image

It works!!!

I obviously had to build it from scratch so it wasn't very straightforward (only missing packages) but IT WORKS! I still need to test out Pyrealsense, the SDK, examples, etc (which compiled successfully).

Also, you are right Asahi Linux is a good match. The amount of work they did is BONKERS. They reverse-engineered firmware for almost of Macs' hardware in a very very short amount of time.

MartyG-RealSense commented 5 months ago

Hi @HasanTheSyrian It's great to hear that you were successful with Asahi Linux! This should be a useful reference for other Mac M1 / M2 users.

HasanTheSyrian commented 5 months ago

Hi @HasanTheSyrian It's great to hear that you were successful with Asahi Linux! This should be a useful reference for other Mac M1 / M2 users.

I was thinking of adding the steps to the repository, but thought I probably can't because it's not officially supported(?)

MartyG-RealSense commented 5 months ago

RealSense users do still submit Mac-related updates as Pull Requests, although it is rare.

https://github.com/IntelRealSense/librealsense/pulls?q=is%3Apr+is%3Aopen+mac

HasanTheSyrian commented 5 months ago

RealSense users do still submit Mac-related updates as Pull Requests, although it is rare.

https://github.com/IntelRealSense/librealsense/pulls?q=is%3Apr+is%3Aopen+mac

Okay Ill add the installation steps to the documentation as a PR soon.

MartyG-RealSense commented 4 months ago

Hi @HasanTheSyrian Have you had the opportunity to submit a PR yet? Thanks!

HasanTheSyrian commented 4 months ago

@MartyG-RealSense I'm still experimenting with Asahi, I haven't forgotten. I'll make a draft within the next 1-3 weeks.

MartyG-RealSense commented 4 months ago

@HasanTheSyrian Thanks very much!

MartyG-RealSense commented 1 month ago

Hi @HasanTheSyrian Did you submit a PR about this case since your last comment on it in February 2024, please? Thanks!

HasanTheSyrian commented 1 month ago

@MartyG-RealSense Yes! Long time ago.

https://github.com/IntelRealSense/librealsense/pull/12763/commits/b9ea8edd611ff2a500ac5e2192319f2f8c49be4a

MartyG-RealSense commented 1 month ago

Thanks very much for the confirmation. I will add an Enhancement label to this case to affirm that it should be kept open whilst your PR is active. Thanks again!