search

NVIDIA-ISAAC-ROS / isaac_ros_nvblox

NVIDIA-accelerated 3D scene reconstruction and Nav2 local costmap provider using nvblox
https://developer.nvidia.com/isaac-ros-gems
Apache License 2.0
416 stars 78 forks source link

issues at jetson #6

Closed AndreV84 closed 1 year ago

AndreV84 commented 2 years ago

you wrote

Therefore, to use nvblox on jetson you have two options:

1. manually compile ROS2 Foxy and required dependent packages from source
2. or use the Isaac ROS development Docker image from [Isaac ROS Common](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common).

However, there is also a third option to do dist-upgrade on jetson that will result in 20.04 ubuntu system-wide AV

AndreV84 commented 2 years ago

what I am getting still is 


NVBlox Timing
-----------
ros/color         1 00.000003   (00.000003 +- 00.000000)    [00.000003,00.000003]
ros/total         1 00.000043   (00.000043 +- 00.000000)    [00.000043,00.000043]

[ERROR] [1650547795.264079730] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547795.422622316] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547795.422805622] [nvblox_node]: Failed to transform color image.
[INFO] [1650547795.599730250] [nvblox_node]: Got a new subscriber, sending entire map.
[ERROR] [1650547796.705232438] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547796.705426464] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547796.705492196] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547797.207929711] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547797.208122841] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547797.208194461] [nvblox_node]: Failed to transform color image.
[ERROR] [1650547797.208260128] [nvblox_node]: Failed to transform color i
Myzhar commented 2 years ago

has it got applied as per the attached output?

It's not applied: [zed_wrapper-2] 1650547554.890821732 [zed.zed_node] [INFO] * Broadcast Pose TF: TRUE

Myzhar commented 2 years ago

Have you added this line? publish_tf: false that's not in the right place. It must be in the pos_tracking sub-group of parameters. You put it in the general subgroup.

You must be aware of what you are doing, otherwise what you do does not work as expected.

AndreV84 commented 2 years ago

@Myzhar Thank you for pointing out! However after I added the false argument in the correct place we got the different errors

ros2 run nvblox_ros nvblox_node --ros-args -r depth/camera_info:=/zed/zed_node/depth/camera_info -r depth/image:=/zed/zed_node/depth/depth_registered -r  color/image:=/zed/zed_node/rgb/image_rect_color -r color/camera_info:=/zed/zed_node/rgb/camera_info -p global_frame:=map
I0421 09:42:46.078943 19276 layer_cake_impl.h:31] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_9TsdfVoxelEEE , and memory_type: kDevice to LayerCake.
I0421 09:42:46.079536 19276 layer_cake_impl.h:31] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_10ColorVoxelEEE , and memory_type: kDevice to LayerCake.
I0421 09:42:46.079569 19276 layer_cake_impl.h:31] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_9EsdfVoxelEEE , and memory_type: kDevice to LayerCake.
I0421 09:42:46.079596 19276 layer_cake_impl.h:31] Adding Layer with type: N6nvblox10BlockLayerINS_9MeshBlockEEE , and memory_type: kDevice to LayerCake.
[INFO] [1650548566.090830837] [nvblox_node]: Outputting results (as requested) to: 
[INFO] [1650548566.091851192] [nvblox_node]: Started up nvblox node in frame map and voxel size 0.05
[INFO] [1650548566.203570554] [nvblox_node]: RGB frame statistics: 
avg=nan, min=nan, max=nan, std_dev=nan, count=0
[INFO] [1650548566.261976289] [nvblox_node]: Depth frame statistics: 
avg=nan, min=nan, max=nan, std_dev=nan, count=0
[INFO] [1650548566.262242034] [nvblox_node]: Timing statistics: 
NVBlox Timing
-----------
ros/color         1 00.000003   (00.000003 +- 00.000000)    [00.000003,00.000003]
ros/total         2 00.000017   (00.000008 +- 00.000007)    [00.000001,00.000016]

Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist

while from zed node the outputs are as follows


ros2 launch zed_wrapper zed.launch.py 
[INFO] [launch]: All log files can be found below /home/nvidia/.ros/log/2022-04-21-09-42-38-132546-agx-19216
[INFO] [launch]: Default logging verbosity is set to INFO
[WARNING] [launch_ros.actions.node]: Parameter file path is not a file: .
[INFO] [robot_state_publisher-1]: process started with pid [19219]
[INFO] [zed_wrapper-2]: process started with pid [19221]
[robot_state_publisher-1] Parsing robot urdf xml string.
[robot_state_publisher-1] Link zed_base_link had 1 children
[robot_state_publisher-1] Link zed_camera_center had 2 children
[robot_state_publisher-1] Link zed_left_camera_frame had 1 children
[robot_state_publisher-1] Link zed_left_camera_optical_frame had 0 children
[robot_state_publisher-1] Link zed_right_camera_frame had 1 children
[robot_state_publisher-1] Link zed_right_camera_optical_frame had 0 children
[robot_state_publisher-1] 1650548558.816967619 [zed.zed_state_publisher] [INFO] got segment base_link
[robot_state_publisher-1] 1650548558.817177809 [zed.zed_state_publisher] [INFO] got segment zed_base_link
[robot_state_publisher-1] 1650548558.817216531 [zed.zed_state_publisher] [INFO] got segment zed_camera_center
[robot_state_publisher-1] 1650548558.817244821 [zed.zed_state_publisher] [INFO] got segment zed_left_camera_frame
[robot_state_publisher-1] 1650548558.817273271 [zed.zed_state_publisher] [INFO] got segment zed_left_camera_optical_frame
[robot_state_publisher-1] 1650548558.817299033 [zed.zed_state_publisher] [INFO] got segment zed_right_camera_frame
[robot_state_publisher-1] 1650548558.817323898 [zed.zed_state_publisher] [INFO] got segment zed_right_camera_optical_frame
[zed_wrapper-2] 1650548558.927365021 [zed.zed_node] [INFO] ********************************
[zed_wrapper-2] 1650548558.927658000 [zed.zed_node] [INFO]       ZED Camera Component 
[zed_wrapper-2] 1650548558.927696851 [zed.zed_node] [INFO] ********************************
[zed_wrapper-2] 1650548558.927728149 [zed.zed_node] [INFO]  * namespace: /zed
[zed_wrapper-2] 1650548558.927788089 [zed.zed_node] [INFO]  * node name: zed_node
[zed_wrapper-2] 1650548558.927815642 [zed.zed_node] [INFO] ********************************
[zed_wrapper-2] 1650548558.927974981 [zed.zed_node] [INFO] *** DEBUG parameters ***
[zed_wrapper-2] 1650548558.928120942 [zed.zed_node] [INFO]  * Debug mode: FALSE
[zed_wrapper-2] 1650548558.928183123 [zed.zed_node] [INFO] *** GENERAL parameters ***
[zed_wrapper-2] 1650548558.928404961 [zed.zed_node] [INFO]  * Camera model: zed - ZED
[zed_wrapper-2] 1650548558.929001288 [zed.zed_node] [INFO]  * SDK Verbose: 1
[zed_wrapper-2] 1650548558.929107919 [zed.zed_node] [INFO]  * SVO: live
[zed_wrapper-2] 1650548558.929177396 [zed.zed_node] [INFO]  * SVO Loop: FALSE
[zed_wrapper-2] 1650548558.929284603 [zed.zed_node] [INFO]  * SVO Realtime: FALSE
[zed_wrapper-2] 1650548558.929355359 [zed.zed_node] [INFO]  * Camera name: zed
[zed_wrapper-2] 1650548558.929739640 [zed.zed_node] [INFO]  * Camera ID: 0
[zed_wrapper-2] 1650548558.930236249 [zed.zed_node] [INFO]  * Camera SN: 0
[zed_wrapper-2] 1650548558.930318206 [zed.zed_node] [INFO]  * Camera timeout [sec]: 5
[zed_wrapper-2] 1650548558.930384291 [zed.zed_node] [INFO]  * Camera reconnection temptatives: 5
[zed_wrapper-2] 1650548558.930483785 [zed.zed_node] [INFO]  * Camera framerate: 15
[zed_wrapper-2] 1650548558.930577839 [zed.zed_node] [INFO]  * GPU ID: -1
[zed_wrapper-2] 1650548558.930690839 [zed.zed_node] [INFO]  * Camera resolution: 2 - HD720
[zed_wrapper-2] 1650548558.931205624 [zed.zed_node] [INFO]  * Camera self calibration: TRUE
[zed_wrapper-2] 1650548558.931287966 [zed.zed_node] [INFO]  * Camera flip: FALSE
[zed_wrapper-2] 1650548558.931374467 [zed.zed_node] [INFO]  * [DYN] Publish framerate [Hz]: 15 
[zed_wrapper-2] 1650548558.931422311 [zed.zed_node] [INFO] *** VIDEO parameters ***
[zed_wrapper-2] 1650548558.931582801 [zed.zed_node] [INFO]  * Use old extrinsic parameters: 0
[zed_wrapper-2] 1650548558.931668791 [zed.zed_node] [INFO]  * [DYN] Image downsample factor: 1 
[zed_wrapper-2] 1650548558.931994476 [zed.zed_node] [INFO]  * [DYN] Brightness: 4
[zed_wrapper-2] 1650548558.932611732 [zed.zed_node] [INFO]  * [DYN] Contrast: 4
[zed_wrapper-2] 1650548558.932756062 [zed.zed_node] [INFO]  * [DYN] Hue: 0
[zed_wrapper-2] 1650548558.932940010 [zed.zed_node] [INFO]  * [DYN] Saturation: 4
[zed_wrapper-2] 1650548558.933033200 [zed.zed_node] [INFO]  * [DYN] Sharpness: 4
[zed_wrapper-2] 1650548558.933095668 [zed.zed_node] [INFO]  * [DYN] Gamma: 8
[zed_wrapper-2] 1650548558.933659385 [zed.zed_node] [INFO]  * [DYN] Auto Exposure/Gain: TRUE
[zed_wrapper-2] 1650548558.933737534 [zed.zed_node] [INFO]  * [DYN] Exposure: 80
[zed_wrapper-2] 1650548558.933852966 [zed.zed_node] [INFO]  * [DYN] Gain: 80
[zed_wrapper-2] 1650548558.933968813 [zed.zed_node] [INFO]  * [DYN] Auto White Balance: TRUE
[zed_wrapper-2] 1650548558.934607095 [zed.zed_node] [INFO]  * [DYN] White Balance Temperature: 42
[zed_wrapper-2] 1650548558.934711390 [zed.zed_node] [INFO]  * Video QoS History: KEEP_LAST
[zed_wrapper-2] 1650548558.934824453 [zed.zed_node] [INFO]  * Video QoS History depth: 1
[zed_wrapper-2] 1650548558.934916235 [zed.zed_node] [INFO]  * Video QoS Reliability: RELIABLE
[zed_wrapper-2] 1650548558.935537076 [zed.zed_node] [INFO]  * Video QoS Durability: VOLATILE
[zed_wrapper-2] 1650548558.935597784 [zed.zed_node] [INFO] *** DEPTH parameters ***
[zed_wrapper-2] 1650548558.935685214 [zed.zed_node] [INFO]  * Depth quality: 1 - PERFORMANCE
[zed_wrapper-2] 1650548558.935820390 [zed.zed_node] [INFO]  * Depth downsample factor: 1 
[zed_wrapper-2] 1650548558.935911020 [zed.zed_node] [INFO]  * Min depth [m]: 0.2
[zed_wrapper-2] 1650548558.935979441 [zed.zed_node] [INFO]  * Max depth [m]: 10
[zed_wrapper-2] 1650548558.936321799 [zed.zed_node] [INFO]  * Depth Sensing Mode: 0 - STANDARD
[zed_wrapper-2] 1650548558.936484146 [zed.zed_node] [INFO]  * Depth Stabilization: TRUE
[zed_wrapper-2] 1650548558.936551126 [zed.zed_node] [INFO]  * OpenNI mode (16bit point cloud): FALSE
[zed_wrapper-2] 1650548558.936624987 [zed.zed_node] [INFO]  * [DYN] Point cloud rate [Hz]: 10
[zed_wrapper-2] 1650548558.936725218 [zed.zed_node] [INFO]  * [DYN] Depth Confidence: 50
[zed_wrapper-2] 1650548558.937164062 [zed.zed_node] [INFO]  * [DYN] Depth Texture Confidence: 100
[zed_wrapper-2] 1650548558.937263589 [zed.zed_node] [INFO]  * [DYN] Remove saturated areas: TRUE
[zed_wrapper-2] 1650548558.937401262 [zed.zed_node] [INFO]  * Depth QoS History: KEEP_LAST
[zed_wrapper-2] 1650548558.937479507 [zed.zed_node] [INFO]  * Depth QoS History depth: 1
[zed_wrapper-2] 1650548558.937542231 [zed.zed_node] [INFO]  * Depth QoS Reliability: RELIABLE
[zed_wrapper-2] 1650548558.937600027 [zed.zed_node] [INFO]  * Depth QoS Durability: VOLATILE
[zed_wrapper-2] 1650548558.937740932 [zed.zed_node] [INFO] *** POSITIONAL TRACKING parameters ***
[zed_wrapper-2] 1650548558.938379566 [zed.zed_node] [INFO]  * Positional tracking enabled: TRUE
[zed_wrapper-2] 1650548558.938524727 [zed.zed_node] [INFO]  * Base frame id: base_link
[zed_wrapper-2] 1650548558.938640959 [zed.zed_node] [INFO]  * Map frame id: map
[zed_wrapper-2] 1650548558.938711172 [zed.zed_node] [INFO]  * Odometry frame id: odom
[zed_wrapper-2] 1650548558.938777224 [zed.zed_node] [INFO]  * Broadcast Odometry TF: FALSE
[zed_wrapper-2] 1650548558.938851821 [zed.zed_node] [INFO]  * [DYN] Path publishing rate: 2
[zed_wrapper-2] 1650548558.938935346 [zed.zed_node] [INFO]  * Path history lenght: -1
[zed_wrapper-2] 1650548558.939199012 [zed.zed_node] [INFO]  * Initial pose: [0,0,0,0,0,0,]
[zed_wrapper-2] 1650548558.939472950 [zed.zed_node] [INFO]  * Area Memory: TRUE
[zed_wrapper-2] 1650548558.939564219 [zed.zed_node] [INFO]  * Area Memory DB: 
[zed_wrapper-2] 1650548558.939633376 [zed.zed_node] [INFO]  * IMU Fusion [not for ZED]: TRUE
[zed_wrapper-2] 1650548558.939698756 [zed.zed_node] [INFO]  * Floor Alignment: FALSE
[zed_wrapper-2] 1650548558.939828973 [zed.zed_node] [INFO]  * Init Odometry with first valid pose data: TRUE
[zed_wrapper-2] 1650548558.940263241 [zed.zed_node] [INFO]  * 2D mode: FALSE
[zed_wrapper-2] 1650548558.940503929 [zed.zed_node] [INFO]  * Pose/Odometry QoS History: KEEP_LAST
[zed_wrapper-2] 1650548558.940577374 [zed.zed_node] [INFO]  * Pose/Odometry QoS History depth: 1
[zed_wrapper-2] 1650548558.940640738 [zed.zed_node] [INFO]  * Pose/Odometry QoS Reliability: RELIABLE
[zed_wrapper-2] 1650548558.940801356 [zed.zed_node] [INFO]  * Pose/Odometry QoS Durability: VOLATILE
[zed_wrapper-2] 1650548558.940843471 [zed.zed_node] [INFO] *** Spatial Mapping parameters ***
[zed_wrapper-2] 1650548558.940909332 [zed.zed_node] [INFO]  * Spatial Mapping Enabled: FALSE
[zed_wrapper-2] 1650548558.940994105 [zed.zed_node] [INFO]  * Spatial Mapping resolution [m]: 0.1
[zed_wrapper-2] 1650548558.941552254 [zed.zed_node] [INFO]  * 3D Max Mapping range [m]: 20
[zed_wrapper-2] 1650548558.941652292 [zed.zed_node] [INFO]  * Map publishing rate [Hz]: 0.5
[zed_wrapper-2] 1650548558.941743114 [zed.zed_node] [INFO]  * Sensors QoS History: KEEP_LAST
[zed_wrapper-2] 1650548558.941804366 [zed.zed_node] [INFO]  * Sensors QoS History depth: 1
[zed_wrapper-2] 1650548558.941862450 [zed.zed_node] [INFO]  * Sensors QoS Reliability: RELIABLE
[zed_wrapper-2] 1650548558.942223946 [zed.zed_node] [INFO]  * Sensors QoS Durability: VOLATILE
[zed_wrapper-2] 1650548558.942507196 [zed.zed_node] [INFO] *** SERVICES ***
[zed_wrapper-2] 1650548558.947562951 [zed.zed_node] [INFO]  * '/zed/zed_node/reset_odometry'
[zed_wrapper-2] 1650548558.949128942 [zed.zed_node] [INFO]  * '/zed/zed_node/reset_pos_tracking'
[zed_wrapper-2] 1650548558.952601425 [zed.zed_node] [INFO]  * '/zed/zed_node/set_pose'
[zed_wrapper-2] 1650548558.953946249 [zed.zed_node] [INFO]  * '/zed/zed_node/enable_obj_det'
[zed_wrapper-2] 1650548558.954656375 [zed.zed_node] [INFO]  * '/zed/zed_node/enable_mapping'
[zed_wrapper-2] 1650548558.955533809 [zed.zed_node] [INFO]  * '/zed/zed_node/start_svo_rec'
[zed_wrapper-2] 1650548558.956251104 [zed.zed_node] [INFO]  * '/zed/zed_node/stop_svo_rec'
[zed_wrapper-2] 1650548558.956892042 [zed.zed_node] [INFO]  * '/zed/zed_node/toggle_svo_pause'
[zed_wrapper-2] 1650548558.956949805 [zed.zed_node] [INFO] ***** STARTING CAMERA *****
[zed_wrapper-2] 1650548558.957014866 [zed.zed_node] [INFO] SDK Version: 3.7.2 - Build 44224_5b6e7702
[zed_wrapper-2] 1650548558.962862064 [zed.zed_node] [INFO] *** CAMERA OPENING ***
[zed_wrapper-2] [ZED][INFO] Logging level INFO
[zed_wrapper-2] [ZED][INFO] [Init]  Depth mode: PERFORMANCE
[zed_wrapper-2] [ZED][INFO] [Init]  Camera successfully opened.
[zed_wrapper-2] [ZED][INFO] [Init]  Camera FW version: 1523
[zed_wrapper-2] [ZED][INFO] [Init]  Video mode: HD720@15
[zed_wrapper-2] [ZED][WARNING] [Init]  Requested depth_minimum_distance: 0.2METER is too close, clamped to 0.3METER.
[zed_wrapper-2] 1650548560.784518890 [zed.zed_node] [INFO] ZED SDK running on GPU #0
[zed_wrapper-2] 1650548560.784808477 [zed.zed_node] [INFO]  * Camera Model  -> ZED
[zed_wrapper-2] 1650548560.784872833 [zed.zed_node] [INFO]  * Serial Number -> -----
[zed_wrapper-2] 1650548560.784968167 [zed.zed_node] [INFO]  * Input type    -> USB input type
[zed_wrapper-2] 1650548560.785012938 [zed.zed_node] [INFO]  * Camera FW Version -> 1523
[zed_wrapper-2] 1650548560.785164340 [zed.zed_node] [INFO] *** TF FRAMES ***
[zed_wrapper-2] 1650548560.785211383 [zed.zed_node] [INFO]  * Map           -> map
[zed_wrapper-2] 1650548560.785271035 [zed.zed_node] [INFO]  * Odometry          -> odom
[zed_wrapper-2] 1650548560.785325406 [zed.zed_node] [INFO]  * Base          -> base_link
[zed_wrapper-2] 1650548560.785360449 [zed.zed_node] [INFO]  * Camera            -> zed_camera_center
[zed_wrapper-2] 1650548560.785393795 [zed.zed_node] [INFO]  * Left          -> zed_left_camera_frame
[zed_wrapper-2] 1650548560.785425285 [zed.zed_node] [INFO]  * Left Optical      -> zed_left_camera_optical_frame
[zed_wrapper-2] 1650548560.785456807 [zed.zed_node] [INFO]  * RGB           -> zed_left_camera_frame
[zed_wrapper-2] 1650548560.785488169 [zed.zed_node] [INFO]  * RGB Optical       -> zed_left_camera_frame
[zed_wrapper-2] 1650548560.785603409 [zed.zed_node] [INFO]  * Right         -> zed_right_camera_frame
[zed_wrapper-2] 1650548560.785640275 [zed.zed_node] [INFO]  * Right Optical     -> zed_right_camera_optical_frame
[zed_wrapper-2] 1650548560.785696983 [zed.zed_node] [INFO]  * Depth         -> zed_left_camera_frame
[zed_wrapper-2] 1650548560.785750874 [zed.zed_node] [INFO]  * Depth Optical     -> zed_left_camera_optical_frame
[zed_wrapper-2] 1650548560.785811582 [zed.zed_node] [INFO]  * Point Cloud       -> zed_left_camera_optical_frame
[zed_wrapper-2] 1650548560.785849057 [zed.zed_node] [INFO]  * Disparity     -> zed_left_camera_frame
[zed_wrapper-2] 1650548560.785884515 [zed.zed_node] [INFO]  * Disparity Optical -> zed_left_camera_optical_frame
[zed_wrapper-2] 1650548560.785916997 [zed.zed_node] [INFO]  * Confidence        -> zed_left_camera_frame
[zed_wrapper-2] 1650548560.785952648 [zed.zed_node] [INFO]  * Confidence Optical    -> zed_left_camera_optical_frame
[zed_wrapper-2] 1650548560.786042797 [zed.zed_node] [INFO] *** PUBLISHED TOPICS ***
[zed_wrapper-2] 1650548560.795907443 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/rgb/image_rect_color
[zed_wrapper-2] 1650548560.797455224 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/rgb/image_rect_gray
[zed_wrapper-2] 1650548560.797568223 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/rgb/camera_info
[zed_wrapper-2] 1650548560.798793392 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/rgb_raw/image_raw_color
[zed_wrapper-2] 1650548560.800075331 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/rgb_raw/image_raw_gray
[zed_wrapper-2] 1650548560.800160041 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/rgb_raw/camera_info
[zed_wrapper-2] 1650548560.801716271 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/left/image_rect_color
[zed_wrapper-2] 1650548560.804317817 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/left/image_rect_gray
[zed_wrapper-2] 1650548560.804409343 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/left/camera_info
[zed_wrapper-2] 1650548560.805548714 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/left_raw/image_raw_color
[zed_wrapper-2] 1650548560.806458725 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/left_raw/image_raw_gray
[zed_wrapper-2] 1650548560.806528138 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/left_raw/camera_info
[zed_wrapper-2] 1650548560.807809533 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/right/image_rect_color
[zed_wrapper-2] 1650548560.809204985 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/right/image_rect_gray
[zed_wrapper-2] 1650548560.809281534 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/right/camera_info
[zed_wrapper-2] 1650548560.810614453 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/right_raw/image_raw_color
[zed_wrapper-2] 1650548560.811862407 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/right_raw/image_raw_gray
[zed_wrapper-2] 1650548560.811931979 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/right_raw/camera_info
[zed_wrapper-2] 1650548560.813086615 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/depth/depth_registered
[zed_wrapper-2] 1650548560.813156283 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/depth/camera_info
[zed_wrapper-2] 1650548560.813864362 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/depth/depth_info
[zed_wrapper-2] 1650548560.814540534 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/stereo/image_rect_color
[zed_wrapper-2] 1650548560.815270598 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/stereo_raw/image_raw_color
[zed_wrapper-2] 1650548560.815771206 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/confidence/confidence_map
[zed_wrapper-2] 1650548560.817911314 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/disparity/disparity_image
[zed_wrapper-2] 1650548560.818789484 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/point_cloud/cloud_registered
[zed_wrapper-2] 1650548560.819679398 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/pose
[zed_wrapper-2] 1650548560.820562752 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/pose_with_covariance
[zed_wrapper-2] 1650548560.822217932 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/odom
[zed_wrapper-2] 1650548560.822825748 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/path_map
[zed_wrapper-2] 1650548560.823240207 [zed.zed_node] [INFO] Advertised on topic: /zed/zed_node/path_odom
[zed_wrapper-2] 1650548560.851621616 [zed.zed_node] [INFO] *** Starting Positional Tracking ***
[zed_wrapper-2] 1650548560.851821501 [zed.zed_node] [INFO]  * Waiting for valid static transformations...
[zed_wrapper-2] 1650548560.852019242 [zed.zed_node] [INFO] Static transform Sensor to Base [zed_left_camera_frame -> base_link]
[zed_wrapper-2] 1650548560.852127505 [zed.zed_node] [INFO]  * Translation: {0.001,-0.060,-0.015}
[zed_wrapper-2] 1650548560.852258041 [zed.zed_node] [INFO]  * Rotation: {0.000,-2.865,0.000}
[zed_wrapper-2] 1650548560.852359904 [zed.zed_node] [INFO] Static transform Sensor to Camera Center [zed_left_camera_frame -> zed_camera_center]
[zed_wrapper-2] 1650548560.852397538 [zed.zed_node] [INFO]  * Translation: {0.000,-0.060,0.000}
[zed_wrapper-2] 1650548560.852433989 [zed.zed_node] [INFO]  * Rotation: {0.000,-0.000,0.000}
[zed_wrapper-2] 1650548560.852480104 [zed.zed_node] [INFO] Static transform Camera Center to Base [zed_camera_center -> base_link]
[zed_wrapper-2] 1650548560.852511146 [zed.zed_node] [INFO]  * Translation: {0.001,0.000,-0.015}
[zed_wrapper-2] 1650548560.852568622 [zed.zed_node] [INFO]  * Rotation: {0.000,-2.865,0.000}
[zed_wrapper-2] 1650548560.952829677 [zed.zed_node] [INFO] Initial ZED left camera pose (ZED pos. tracking): 
[zed_wrapper-2] 1650548560.952969942 [zed.zed_node] [INFO]  * T: [-1.66741e-19,0.06,0.015]
[zed_wrapper-2] 1650548560.953047419 [zed.zed_node] [INFO]  * Q: [0,0.0249967,0,0.999687]
[zed_wrapper-2] 1650548561.109792136 [zed.zed_node] [INFO] Odometry aligned to last tracking pose
AndreV84 commented 2 years ago

now zed node output shows rather 

[zed_wrapper-2] 1650548558.938777224 [zed.zed_node] [INFO]  * Broadcast Odometry TF: FALSE

than the previous output before adding the false argument which was


 [INFO] * Broadcast Pose TF: TRUE
AndreV84 commented 2 years ago 
ros2 run tf2_tools view_frames.py
[INFO] [1650549111.133168235] [view_frames]: Listening to tf data during 5 seconds...
[INFO] [1650549116.160482297] [view_frames]: Generating graph in frames.pdf file...
[INFO] [1650549116.167002047] [view_frames]: Result:tf2_msgs.srv.FrameGraph_Response(frame_yaml="zed_base_link: \n  parent: 'base_link'\n  broadcaster: 'default_authority'\n  rate: 10000.000\n  most_recent_transform: 0.000000\n  oldest_transform: 0.000000\n  buffer_length: 0.000\nzed_camera_center: \n  parent: 'zed_base_link'\n  broadcaster: 'default_authority'\n  rate: 10000.000\n  most_recent_transform: 0.000000\n  oldest_transform: 0.000000\n  buffer_length: 0.000\nzed_left_camera_frame: \n  parent: 'zed_camera_center'\n  broadcaster: 'default_authority'\n  rate: 10000.000\n  most_recent_transform: 0.000000\n  oldest_transform: 0.000000\n  buffer_length: 0.000\nzed_left_camera_optical_frame: \n  parent: 'zed_left_camera_frame'\n  broadcaster: 'default_authority'\n  rate: 10000.000\n  most_recent_transform: 0.000000\n  oldest_transform: 0.000000\n  buffer_length: 0.000\nzed_right_camera_frame: \n  parent: 'zed_camera_center'\n  broadcaster: 'default_authority'\n  rate: 10000.000\n  most_recent_transform: 0.000000\n  oldest_transform: 0.000000\n  buffer_length: 0.000\nzed_right_camera_optical_frame: \n  parent: 'zed_right_camera_frame'\n  broadcaster: 'default_authority'\n  rate: 10000.000\n  most_recent_transform: 0.000000\n  oldest_transform: 0.000000\n  buffer_length: 0.000\n")

frames.pdf

AndreV84 commented 2 years ago

now the false argument impacted the output of the command below


 ros2 run tf2_ros tf2_echo map zed_left_camera_frame
[INFO] [1650549405.588226037] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[INFO] [1650549407.501974300] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[INFO] [1650549408.501976058] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[INFO] [1650549409.502040952] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[INFO] [1650549410.502199513] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[INFO] [1650549412.501943503] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
^C[INFO] [1650549413.334958408] [rclcpp]: signal_hand
helenol commented 2 years ago

For context nvblox just consumes the tf tree from global frame (map in this case I believe) to the camera optical frames, which is working in @AndreV84's setup just fine for the depth map but for some reason not for the color image. We pull the frame_id for both off the image's header. We do not publish any additional tf frames.

AndreV84 commented 2 years ago

@helenol Thank you for the update how could I manually try to consume tf tree from global frame in the same manner the nvblox code does it? so there will be either indicative error or opposite? Thanks AV

AndreV84 commented 2 years ago

@Myzhar After I remove the proposed :false modification I can again run this command successfully


ros2 run tf2_ros tf2_echo map zed_left_camera_frame
[INFO] [1650550602.236420099] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
At time 1650550603.113246982
- Translation: [-0.000, 0.060, 0.015]
- Rotation: in Quaternion [0.000, 0.025, 0.000, 1.000]
At time 1650550604.113308982
- Translation: [-0.000, 0.060, 0.015]
- Rotation: in Quaternion [0.000, 0.025, -0.000, 1.000]
AndreV84 commented 2 years ago

so as a user I am getting either this from the app

[ERROR] [1650550823.508767149] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.535148770] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.535341069] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.734482134] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.734647327] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.734713443] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.934379946] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.934567252] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.934637592] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550823.934697084] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550824.201481679] [nvblox_node]: Failed to transform color image.
[ERROR] [1650550824.201712956] [nvblox_node]: Failed to transform color image.

and this

1652489] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
At time 1650551221.92225696
- Translation: [-0.000, 0.060, 0.015]
- Rotation: in Quaternion [0.000, 0.025, -0.000, 1.000]

or If I add the proposed :false modification, I will get this

Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
         at line 133 in /tmp/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp

and this

ros2 run tf2_ros tf2_echo map zed_left_camera_frame
[INFO] [1650551157.785110594] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[INFO] [1650551159.746870841] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[INFO] [1650551160.746891854] [tf2_echo]: Waiting for transform map ->  zed_left_camera_frame: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist

if we disable the broadcast of TF as @Myzhar proposed

helenol commented 2 years ago

@AndreV84 can you set publish TF to true and record me a rosbag? With /tf /tf_static and all the image topics we've discussed above (color, depth, and camera_info for both color and depth).

Guide: https://docs.ros.org/en/foxy/Tutorials/Ros2bag/Recording-And-Playing-Back-Data.html#ros2-bag-record Example:

ros2 bag record /tf /tf_static /whatever/your/image/topics/were /and/some/other/topics/here
AndreV84 commented 2 years ago

@helenol you may download recorded rosbag using the url below https://andrev.file.core.windows.net/nvidiazed/rosbag2_2022_04_21-11_52_39_0.db3?sp=rl&st=2022-04-21T16:29:52Z&se=2022-04-22T16:29:52Z&sv=2020-08-04&sig=71lQBfs1WLlkAUYFPQ5yYyAJDen8jjmaoPozTsEFZU4%3D&sr=f

AndreV84 commented 2 years ago

@helenol WERE YOU ABLE TO DOWNLOAD THE ROSBAG?

cjh1995-ros commented 2 years ago

Did you solve the problem? I am also trying to use realsense d435i and ZED2, but I have encountered the same problem as you.

For the d435i, it looks like this: 2

In the case of ZED2, the depth mode was changed from performance to nueral, so the picture changed as follows.
1 3

But I still don't think I can get the color image from ZED2. If you solved the problem, please share it!

cjh1995-ros commented 2 years ago

In the picture above, the last image is a picture of both zed's pcl and nvblox mesh. The mesh is not colored.

AndreV84 commented 2 years ago

@cjh1995-ros Thank you for following up! Unfortunately I was not able to get the color from ZED. @helenol @Myzhar used to try helping out with that but they seem gave up on supporting it

cjh1995-ros commented 2 years ago

@AndreV84 Thank you for your reply. Too bad. I also wanted to use this library easily, but I still have a long way to go. From now on, I'm going to touch whether the parameters are important or not. If you have any experience, can you share it with me?

AndreV84 commented 2 years ago

@cjh1995-ros Nihao Xie Xie

cjh1995-ros commented 2 years ago

First, I'm not a Chinese. I'm from Korea. LOL

Both of them! In nvblox param folder, I saw the color hz parameter. So next Monday, I'm going to test that.

Thank you for reply!

AndreV84 commented 2 years ago

haha, we shall see I do not even know where is nvblox param folder at this github repository However we could use ros topic hz to show color topic information in terminal e.g. for diagnostics as in


ros2 topic hz /zed/zed_node/rgb/image_rect_color
cjh1995-ros commented 2 years ago

image

It's in here!

I didn't test that part. So have to keep going on.

Thanks for reply!

Aki1608 commented 2 years ago

@cjh1995-ros Were you able to get good output from camera? I am trying with realsense d435i camera. I also got similar output as yours and it takes a bit of time to create the output as well. Is it taking more time for you to generate output as well?

In isaac sim, it works way better. But with camera it is not giving that good of an output. It's really hard to identify the object from the 3d output of the camera. Is it normal behavior from nvblox? Or it should give good 3d output as we are getting from isaac sim?

cjh1995-ros commented 2 years ago

@cjh1995-ros Were you able to get good output from camera? I am trying with realsense d435i camera. I also got similar output as yours and it takes a bit of time to create the output as well. Is it taking more time for you to generate output as well?

In isaac sim, it works way better. But with camera it is not giving that good of an output. It's really hard to identify the object from the 3d output of the camera. Is it normal behavior from nvblox? Or it should give good 3d output as we are getting from isaac sim?

First of all, I got some decent pixels by tuning the parameters. I recommend you to reduce the voxel size first.

Aki1608 commented 2 years ago

@cjh1995-ros

Yes. I tried parameter tuning and got better output.

Another thing that I tried was the preset config file from intel realsense. Intel has provided some preset config file for "high density" , "high accuracy" , etc. visual presets.

I loaded that config file at the time of running nvblox and I was able to get better 3d map than before. Specifically "high density" and "medium density".

AndreV84 commented 1 year ago

@helenol @hemalshahNV So any chance to get the color? @cjh1995-ros could you share the execution parameters used to run ros with using zed2 please?