NVIDIA-ISAAC-ROS / isaac_ros_image_pipeline

NVIDIA-accelerated ROS2 packages for camera image processing.
https://developer.nvidia.com/isaac-ros-gems
Apache License 2.0
112 stars 19 forks source link

no pointcloud message published #22

Closed MrOCW closed 1 year ago

MrOCW commented 1 year ago

image Hi, I have a disparity image but when I run ros2 topic hz /points2, there is no output.

image

I attempted to use the zed2 camera's disparity image instead of isaac ros disparity image for the pointcloud and still, theres no output

[component_container-1] [INFO] [1671096454.236553058] [disparity.disparity_container]: Load Library: /isaac_ros/install/isaac_ros_stereo_image_proc/lib/libpoint_cloud_node.so
[component_container-1] [INFO] [1671096454.290832296] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container-1] [INFO] [1671096454.291127438] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::nitros::NitrosNode>
[component_container-1] [INFO] [1671096454.291191471] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::stereo_image_proc::PointCloudNode>
[component_container-1] [INFO] [1671096454.291204752] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::stereo_image_proc::PointCloudNode>
[component_container-1] [INFO] [1671096454.298994417] [NitrosNode]: [NitrosNode] Initializing NitrosNode
[component_container-1] [INFO] [1671096454.299896196] [NitrosNode]: [NitrosNode] Starting NitrosNode
[component_container-1] [INFO] [1671096454.299940485] [NitrosNode]: [NitrosNode] Loading built-in preset extension specs
[component_container-1] [INFO] [1671096454.302319926] [NitrosNode]: [NitrosNode] Loading built-in extension specs
[component_container-1] [INFO] [1671096454.302443417] [NitrosNode]: [NitrosNode] Loading preset extension specs
[component_container-1] [INFO] [1671096454.303043941] [NitrosNode]: [NitrosNode] Loading extension specs
[component_container-1] [INFO] [1671096454.303087686] [NitrosNode]: [NitrosNode] Loading generator rules
[component_container-1] [INFO] [1671096454.303101542] [NitrosNode]: [NitrosNode] Loading extensions
[component_container-1] [INFO] [1671096454.303505551] [NitrosNode]: [NitrosContext] Loading extension: gxf/std/libgxf_std.so
[component_container-1] [INFO] [1671096454.306754354] [NitrosNode]: [NitrosContext] Loading extension: gxf/multimedia/libgxf_multimedia.so
[component_container-1] [INFO] [1671096454.307192507] [NitrosNode]: [NitrosContext] Loading extension: gxf/cuda/libgxf_cuda.so
[component_container-1] [INFO] [1671096454.308719227] [NitrosNode]: [NitrosContext] Loading extension: gxf/serialization/libgxf_serialization.so
[component_container-1] [INFO] [1671096454.310751653] [NitrosNode]: [NitrosContext] Loading extension: gxf/libgxf_point_cloud.so
[component_container-1] [INFO] [1671096454.360456748] [NitrosNode]: [NitrosContext] Loading extension: gxf/libgxf_synchronization.so
[component_container-1] [INFO] [1671096454.361779623] [NitrosNode]: [NitrosNode] Loading graph to the optimizer
[component_container-1] [INFO] [1671096454.365073195] [NitrosNode]: [NitrosNode] Running optimization
[component_container-1] [INFO] [1671096454.401619393] [NitrosNode]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container-1] [INFO] [1671096454.402546580] [NitrosNode]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container-1] [INFO] [1671096454.415402399] [NitrosNode]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/NitrosNode' in container '/disparity/disparity_container'
[component_container-1] [INFO] [1671096454.532808449] [NitrosContext]: [NitrosContext] Loading application: '/isaac_ros/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml'
[component_container-1] [INFO] [1671096454.533977594] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096454.593242694] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096454.594240987] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096454.718856371] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096454.799840418] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096454.925754421] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096454.987113326] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096454.998402840] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.004442005] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.005244262] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.130424842] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.131536513] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.198594895] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.199588740] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.331221614] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.332248131] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.400106979] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.401147224] [NitrosNode]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container-1] [INFO] [1671096455.416188848] [NitrosNode]: [NitrosNode] Starting post negotiation setup
[component_container-1] [INFO] [1671096455.416289618] [NitrosNode]: [NitrosNode] Getting data format negotiation results
[component_container-1] [INFO] [1671096455.416315859] [NitrosNode]: [NitrosPublisher] Negotiation failed
[component_container-1] [INFO] [1671096455.416329267] [NitrosNode]: [NitrosPublisher] Use only the compatible publisher: topic_name="/points2", data_format="nitros_point_cloud"
[component_container-1] [INFO] [1671096455.416344659] [NitrosNode]: [NitrosSubscriber] Negotiation failed
[component_container-1] [INFO] [1671096455.416354291] [NitrosNode]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/zed2i/zed_node/disparity/disparity_image", data_format="nitros_disparity_image_32FC1"
[component_container-1] [INFO] [1671096455.416364276] [NitrosNode]: [NitrosSubscriber] Negotiation failed
[component_container-1] [INFO] [1671096455.416371316] [NitrosNode]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/zed2i/zed_node/left_rgb/image_rect_color", data_format="nitros_image_bgr8"
[component_container-1] [INFO] [1671096455.416380052] [NitrosNode]: [NitrosSubscriber] Negotiation failed
[component_container-1] [INFO] [1671096455.416387028] [NitrosNode]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/left/camera_info", data_format="nitros_camera_info"
[component_container-1] [INFO] [1671096455.416394740] [NitrosNode]: [NitrosSubscriber] Negotiation failed
[component_container-1] [INFO] [1671096455.416401012] [NitrosNode]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/right/camera_info", data_format="nitros_camera_info"
[component_container-1] [INFO] [1671096455.416481334] [NitrosNode]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container-1] [INFO] [1671096455.466887531] [NitrosNode]: [NitrosNode] Wrote the final top level YAML graph to "/isaac_ros/install/isaac_ros_stereo_image_proc/share/isaac_ros_stereo_image_proc/CQWAUQOGNS.yaml"
[component_container-1] [INFO] [1671096455.466981037] [NitrosNode]: [NitrosNode] Calling user's pre-load-graph callback
[component_container-1] [INFO] [1671096455.467007054] [NitrosNode]: [PointCloudNode] preLoadGraphCallback().
[component_container-1] [INFO] [1671096455.467041647] [NitrosNode]: [NitrosNode] Loading application
[component_container-1] [INFO] [1671096455.467057903] [NitrosNode]: [NitrosContext] Loading application: '/isaac_ros/install/isaac_ros_stereo_image_proc/share/isaac_ros_stereo_image_proc/CQWAUQOGNS.yaml'
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'disparity_image_input' in component ''.
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'rgb_image_input' in component ''.
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'left_cam_input' in component ''.
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'right_cam_input' in component ''.
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'disparity_image_output' in component ''.
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'rgb_image_output' in component ''.
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'left_cam_output' in component ''.
[component_container-1] 2022-12-15 09:27:35.469 WARN  gxf/std/yaml_file_loader.cpp@459: Using unregistered parameter 'right_cam_output' in component ''.
[component_container-1] [INFO] [1671096455.469809864] [NitrosNode]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container-1] [INFO] [1671096455.469913802] [NitrosNode]: [NitrosNode] Calling user's post-load-graph callback
[component_container-1] [INFO] [1671096455.469967883] [NitrosNode]: [NitrosContext] Initializing applicaiton...
[component_container-1] [INFO] [1671096455.517049244] [NitrosNode]: [NitrosContext] Running appliation...
hemalshahNV commented 1 year ago

Could you post a snippet of your launch file? It looks like disparity is fine but PointCloudNode is not running correctly and receiving the disparity for publishing to /points2. It is not clear why PointCloudNode appears as "/NitrosNode" in rQT either.

MrOCW commented 1 year ago
def generate_launch_description():
    disparity_node = ComposableNode(
        name='disparity',
        package='isaac_ros_stereo_image_proc',
        plugin='nvidia::isaac_ros::stereo_image_proc::DisparityNode',
        parameters=[{
                'backends': 'CUDA',
                'max_disparity': 64.0,
        }],
        remappings=[('/left/image_rect', '/zed2i/zed_node/left_rgb/image_rect_color'),
                    ('/right/image_rect','/zed2i/zed_node/right_rgb/image_rect_color'),
                    ('/left/camera_info','/zed2i/zed_node/left/camera_info'),
                    ('/right/camera_info','/zed2i/zed_node/right/camera_info'),
                    ]
        )

    pointcloud_node = ComposableNode(
        package='isaac_ros_stereo_image_proc',
        plugin='nvidia::isaac_ros::stereo_image_proc::PointCloudNode',
        parameters=[{
                'use_color': False,
        }],
        remappings=[('/left/image_rect_color', '/zed2i/zed_node/left_rgb/image_rect_color'),
                    ('/disparity','/zed2i/zed_node/disparity/disparity_image')])

    container = ComposableNodeContainer(
        name='disparity_container',
        namespace='disparity',
        package='rclcpp_components',
        executable='component_container',
        composable_node_descriptions=[pointcloud_node],
        output='screen'
    )

Not launching disparity as i'm using the one from ZED.

hemalshahNV commented 1 year ago

Could you also post the part where you're launching the ZED node? There may be something about the disparity image that ZED is publishing that is incompatible with PointCloudNode. If you can post the output (truncated of course) of ros2 topic echo on the topic for ZED's disparity image: /zed2i/zed_node/disparity/disparity_image, that would be helpful to see if different fields are populated than we expected.