search

IntelRealSense / realsense-ros

ROS Wrapper for Intel(R) RealSense(TM) Cameras
http://wiki.ros.org/RealSense
Apache License 2.0
2.52k stars 1.74k forks source link

Running D435i Camera in ROS and DeepStream SDK simultaneously #1620

Closed MrOCW closed 3 years ago

MrOCW commented 3 years ago

Hi @doronhi ,

Before using roslaunch realsense2_camera opensource_tracking.launch, when i type cd /dev , followed by ls, the files /dev/video0 , /dev/video1 , /dev/video2 , /dev/video3 , /dev/video4 , /dev/video5, appears.

However, after running roslaunch realsense2_camera opensource_tracking.launch, the /dev/video1,2,3,4,5 disappears and I am then unable to continue using them for streaming in NVIDIA's DeepStream SDK.

Is there any way to use the camera with the launch file but still retain the /dev/video1,2,3,4,5?

Thank you!

MartyG-RealSense commented 3 years ago

Hi @MrOCW I researched your question carefully, and multiple sources stated that video0 is used for video capture, but video1 is just used for providing camera metadata about video0 (not a video capture).

https://unix.stackexchange.com/a/539573

Also, may I confirm that when you say that you have a 'USB camera', do you mean that literally - an ordinary USB video webcam that is not an Intel RealSense depth sensing camera?

MrOCW commented 3 years ago

Hi @MrOCW I researched your question carefully, and multiple sources stated that video0 is used for video capture, but video1 is just used for providing camera metadata about video0 (not a video capture).

https://unix.stackexchange.com/a/539573

Also, may I confirm that when you say that you have a 'USB camera', do you mean that literally - an ordinary USB video webcam that is not an Intel RealSense depth sensing camera?

Sorry @MartyG-RealSense , i meant that all the /dev/video disappears, including /dev/video0, when i run roslaunch.

it is a D435i camera

MartyG-RealSense commented 3 years ago

If we think of DeepStream SDK as being a Linux video tool, then building librealsense with the V4L2 backend could potentially grant access to the camera stream.

https://github.com/IntelRealSense/librealsense/issues/6841#issuecomment-660859774

Are you already able to access a stream from the camera in DeepStream SDK if you do not run roslaunch first though?

MrOCW commented 3 years ago

If we think of DeepStream SDK as being a Linux video tool, then building librealsense with the V4L2 backend could potentially grant access to the camera stream.

IntelRealSense/librealsense#6841 (comment)

Are you already able to access a stream from the camera in DeepStream SDK if you do not run roslaunch first though?

Yup! I ran an object detection model in a live stream from the D435i camera, it works fine. Then i proceeded to launch roslaunch realsense2_camera opensource_tracking.launch, which works just fine. However, this stops the DeepStream live stream and the /dev/video0,1,2,3,4,5 disappears from the /dev folder which i suspect might be the reason why DeepStream stops

MartyG-RealSense commented 3 years ago

Thanks very much for the information. In order to test whether the issue is specific to the roslaunch realsense2_camera opensource_tracking.launch roslaunch instruction that you are using, could you try the roslaunch instruction below and see whether video0 to video5 also disappear when using that launch method, please?

roslaunch realsense2_camera rs_camera.launch

MrOCW commented 3 years ago

Thanks very much for the information. In order to test whether the issue is specific to the roslaunch realsense2_camera opensource_tracking.launch roslaunch instruction that you are using, could you try the roslaunch instruction below and see whether video0 to video5 also disappear when using that launch method, please?

roslaunch realsense2_camera rs_camera.launch

Yes @MartyG-RealSense video0 to video5 still disappears after using roslaunch realsense2_camera rs_camera.launch

MartyG-RealSense commented 3 years ago

I conducted further research but did not find a case that is simiar to your problem. I will therefore link @doronhi the RealSense ROS wrapper developer into this case to seek advice.

Hi @doronhi In this case, @MrOCW is finding that dev/video0 to video5 disappear when roslaunch is performed. This prevents them from using video0 to video5 in the DeepStream SDK. Do you have thoughts about why this may be occurring please? Thanks!

MartyG-RealSense commented 3 years ago

Hi @MrOCW Do you require further assistance with this case, please? Thanks!

MrOCW commented 3 years ago

Hi @MrOCW Do you require further assistance with this case, please? Thanks!

Hi @MartyG-RealSense, yes. Unfortunately, issue has not been resolved,

MartyG-RealSense commented 3 years ago

I will send a reminder to @doronhi

Hi @doronhi In this case, @MrOCW is finding that dev/video0 to video5 disappear when roslaunch is performed. This prevents them from using video0 to video5 in the DeepStream SDK. Do you have thoughts about why this may be occurring please? Thanks!

doronhi commented 3 years ago

Sorry, I don't have any ideas. I find it most likely that such an issue concerns the underlying librealsense2 library rather then the realsense2_camera wrapper. However, in order to investigate further I would have to know the kernel version and the kernel logs from around the time of the device disappearance. I am not familiar with DeepStream SDK. From The NVidia introduction I understand it to be a layer between C++/python and CUDA. Where is it used in the context of realsense2_camera ROS node? Can you simplify the case and remove the DeepStream SDK from the equation? And on the same course of investigation, can we remove ROS from the equation as well? What happens when you run realsense-viewer?

MrOCW commented 3 years ago

Sorry, I don't have any ideas. I find it most likely that such an issue concerns the underlying librealsense2 library rather then the realsense2_camera wrapper. However, in order to investigate further I would have to know the kernel version and the kernel logs from around the time of the device disappearance.

May I know how i can obtain those?

I am not familiar with DeepStream SDK. From The NVidia introduction I understand it to be a layer between C++/python and CUDA. Where is it used in the context of realsense2_camera ROS node? Can you simplify the case and remove the DeepStream SDK from the equation?

I am trying to use the same D435i camera for both AI inference from DeepStream (which requires dev/video0-5 and is not related to the ROS nodes), and for the RTabMap SLAM as well (which seemingly removes dev/video0-5).

And on the same course of investigation, can we remove ROS from the equation as well? What happens when you run realsense-viewer?

the dev/video0-5 disappears when i run realsense-viewer

MartyG-RealSense commented 3 years ago

Hi @MrOCW Do you still require assistance with this case, please? Thanks!

MartyG-RealSense commented 3 years ago

Case closed due to no further comments received.