Closed zoidzilla closed 3 years ago
ooeygui
commented
4 years ago Thank you for the report. It looks like it is failing in the SDK. This failure can be caused by many reasons - USB contention, CPU/GPU starvation, resource conflicts, etc.
What other ROS components are running? If they share the USB bus, that could be a source of failure.
zoidzilla
commented
4 years ago Hello,
Yes indeed it is failing in the SDK. To test if there was any conflict with any other components (e.g. laser scanners). I commented out all components in the boot up launch file leaving only the part where I call the Azure camera launch file. However, I still received the same behavior. I also tested different USB ports on NUC as well as different usb cables but the problem remained. Since the problem appears only when I try to launch the Azure Kinect driver from the boot up launch file and not when I am launching it separately in a new terminal, I tried to use the timed_roslaunch package (http://wiki.ros.org/timed_roslaunch) where I put a 15 sec delay for launching the driver in the boot up. However the problem still remains at large. I don't know if you have any other suggestion, but this is a super bug that I have not encountered in the past.
Let me know if you have any additional ideas.
Thank you
ooeygui
commented
4 years ago Can you tell me more about the boot time script? Can you share it? And how you schedule it?
ooeygui
commented
4 years ago I am following up with the Azure Kinect team.
Can you try launching the azure kinect viewer in your startup script instead of ROS? I'd like to see if that is a permissions or access issue.
zoidzilla
commented
4 years ago Hello,
Thank you for the feedback. I will come back with new information and with testing your suggestions as soon as I go back to the lab.
Best regards.
zoidzilla
commented
4 years ago Hello,
We finally found the problem. The problem was that the SDK seems to require a display connected to the computer, otherwise the driver fails to start the camera.
The main problem was to start the camera headless (i.e. without a monitor):
I.e. essentially using nodm to start a virtual display, and updating our systemd service to run after the nodm service.
This is obviously a problem in the SDK and it really depreciates Kinect's value, which otherwise is one of the best RGB-D sensors I have ever worked with. Having said that, it would be extremely important if Microsoft's team take actions to overcome this issue. One last thing I want to try, which maybe simplifies things, is to use a dummy monitor plug (https://www.amazon.de/-/en/Headless-Display-Virtual-Emulator-Generation/dp/B076WP9RZ1/ref=sr_1_fkmr2_2?dchild=1&keywords=4k+hdmi+blindstecker+-+hochaufl%C3%B6sende+virtual+bildschirmfl%C3%A4che+emulator%2C+new+generation+headless+display+adapter&qid=1602710811&sr=8-2-fkmr2) and see if the issue is resolved.
In any case thank you for your response!
Best regards,
zoidzilla
commented
3 years ago Hello,
I would like to verify that the solution of the dummy HDMI monitor plug (https://www.amazon.de/-/en/Headless-Display-Virtual-Emulator-Generation/dp/B076WP9RZ1/ref=sr_1_fkmr2_2?dchild=1&keywords=4k+hdmi+blindstecker+-+hochaufl%C3%B6sende+virtual+bildschirmfl%C3%A4che+emulator%2C+new+generation+headless+display+adapter&qid=1602710811&sr=8-2-fkmr2) in combination with the instructions under section 'Using Azure Kinect on headless Linux system' in https://docs.microsoft.com/en-us/azure/kinect-dk/troubleshooting, resolve the issue.
I think we can close this conversation with the comments above.
Best regards,
Hello,
I am experiencing a very strange behavior with the Kinect Azure ROS driver launch file. I am trying to launch the driver together with other modules in an automatic boot up launch file from a mobile robot and I am constantly getting the following error:
● mp-ros-bootup.service - "Bootup service for mobile platform core components" Loaded: loaded (/etc/systemd/system/mp-ros-bootup.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2020-10-12 11:27:11 CEST; 5s ago Main PID: 17354 (mp-ros-boot) Tasks: 92 (limit: 4915) CGroup: /system.slice/mp-ros-bootup.service ├─17354 /bin/bash /usr/sbin/mp-ros-bootup ├─17395 /usr/bin/python /opt/ros/melodic/bin/roslaunch mp_bringup bringup.launch prefix:= use_case:=omni_4sw_v2 ├─17430 /usr/bin/python /opt/ros/melodic/bin/rosmaster --core -p 11311 -w 3 log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec1631fe/master.log ├─17452 /opt/ros/melodic/lib/rosout/rosout name:=rosout log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec1631fe/rosout-1.log ├─17459 /home/atuin2/Documents/core_ws/devel/lib/mp_driver/driver_node __name:=driver_node log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec1631fe/driver_node-2.log ├─17460 python /opt/ros/melodic/lib/controller_manager/spawner mp_feedback_controller mp_command_twist_controller name:=controller_manager_initial_started log:=/root/.ros/log/1bb2fe68-0c6d- ├─17462 python /opt/ros/melodic/lib/controller_manager/spawner --stopped mp_command_stop_controller name:=controller_manager_initial_stopped log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329 ├─17472 /opt/ros/melodic/lib/robot_state_publisher/robot_state_publisher joint_states:=feedback/joint_states name:=robot_state_publisher log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec16 ├─17473 /opt/ros/melodic/lib/joy/joy_node name:=joy_node __log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec1631fe/bluetooth_teleop-joy_node-6.log ├─17479 /home/atuin2/Documents/core_ws/devel/lib/teleop_twist_joy/teleop_node cmd_vel:=/mp_command_twist_controller/twist_command name:=teleop_twist_joy log:=/root/.ros/log/1bb2fe68-0c6d-11 ├─17486 /home/atuin2/Documents/core_ws/devel/lib/urg_node/urg_node /scan:=rear/scan name:=urg_node_rear log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec1631fe/urg_node_rear-8.log ├─17492 /home/atuin2/Documents/core_ws/devel/lib/urg_node/urg_node /scan:=front/scan name:=urg_node_front log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec1631fe/urg_node_front-9.log ├─17503 /home/atuin2/Documents/core_ws/devel/lib/rc_status/status_node status_node/joint_states:=feedback/joint_states name:=status_node log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329 └─17505 /home/atuin2/Documents/core_ws/devel/lib/rc_command/command_node __name:=command_node log:=/root/.ros/log/1bb2fe68-0c6d-11eb-b73c-2329ec1631fe/rc-command_node-12.log
okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: [2020-10-12 11:27:14.932] [error] [t=17581] /w/1/s/extern/Azure-Kinect-Sensor-SDK/src/dewrapper/dewrapper.c (194): depth_engine_start_helper(de okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: [2020-10-12 11:27:14.932] [error] [t=17496] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/dewrapper/dewrapper.c (552): dewrapper_start(). Depth Eng okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: [2020-10-12 11:27:14.932] [error] [t=17496] /w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth/depth.c (398): dewrapper_start(depth->dewrapper, co okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: [2020-10-12 11:27:14.933] [error] [t=17496] /w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (359): cmd_status == CMD_STATUS_PAS okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: [2020-10-12 11:27:14.933] [error] [t=17496] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (362): depthmcu_depth_stop_streamin okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: [2020-10-12 11:27:14.933] [error] [t=17496] /w/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (895): depth_start(device->depth, config) retur okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: terminate called after throwing an instance of 'k4a::error' okt 12 11:27:14 atuin2-NUC8i7BEH mp-ros-bootup[17354]: what(): Failed to start cameras!
On the other hand, and for the same setup, when I comment out the driver's launch file from the boot up service and I launch the driver individually in another terminal (after the boot up), everything seem to be working fine and the driver launches correctly.
The parameters I am currently having on the driver's launch file are the following:
arg name="depth_enabled" default="true",
arg name="depth_mode" default="NFOV_UNBINNED",
arg name="color_enabled" default="true",
arg name="color_format" default="bgra",
arg name="color_resolution" default="720P",
arg name="fps" default="30",
arg name="point_cloud" default="true",
arg name="rgb_point_cloud" default="true",
arg name="point_cloud_in_depth_frame" default="false",
arg name="required" default="false",
arg name="sensor_sn" default="",
arg name="recording_file" default="",
arg name="recording_loop_enabled" default="false",
arg name="body_tracking_enabled" default="false",
arg name="body_tracking_smoothing_factor" default="0.0",
arg name="rescale_ir_to_mono8" default="false",
arg name="ir_mono8_scaling_factor" default="1.0",
arg name="imu_rate_target" default="0",
arg name="wired_sync_mode" default="0",
arg name="subordinate_delay_off_master_usec" default="0".
My Setup is: Computer: NUC8i7BEH OS: Ubuntu 18.04 ROS: Melodic Azure SDK: 1.4 (installed from binaries as $ sudo apt install libk4a1.4-dev ).
I would appriciate any help in this issue because it is truly mind boggling.
Thank you!