search

NVIDIA-ISAAC-ROS / isaac_ros_compression

NVIDIA-accelerated data compression
https://developer.nvidia.com/isaac-ros-gems
Apache License 2.0
53 stars 7 forks source link

Encoder node never starts #2

Closed ad-cobot closed 1 year ago

ad-cobot commented 1 year ago

I'm trying to run the encoder on an orin dev kit and it seems to crash with some sort of internal gxf library failure. Below are the logs that I get for the command ros2 launch isaac_ros_h264_encoder isaac_ros_h264_encoder.launch.py which always fails.

[INFO] [component_container-1]: process started with pid [17834]
[component_container-1] [INFO] [1678834616.339717362] [encoder_container]: Load Library: /install/isaac_ros_h264_encoder/lib/libencoder_node.so
[component_container-1] [INFO] [1678834616.430238738] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container-1] [INFO] [1678834616.431006067] [encoder_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::h264_encoder::EncoderNode>
[component_container-1] [INFO] [1678834616.431097523] [encoder_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::h264_encoder::EncoderNode>
[component_container-1] [INFO] [1678834616.443703050] [encoder]: [NitrosNode] Initializing NitrosNode
[component_container-1] [INFO] [1678834616.448715987] [encoder]: [NitrosNode] Starting NitrosNode
[component_container-1] [INFO] [1678834616.448781875] [encoder]: [NitrosNode] Loading built-in preset extension specs
[component_container-1] [INFO] [1678834616.451314039] [encoder]: [NitrosNode] Loading built-in extension specs
[component_container-1] [INFO] [1678834616.451386199] [encoder]: [NitrosNode] Loading preset extension specs
[component_container-1] [INFO] [1678834616.452137881] [encoder]: [NitrosNode] Loading extension specs
[component_container-1] [INFO] [1678834616.452179225] [encoder]: [NitrosNode] Loading generator rules
[component_container-1] [INFO] [1678834616.452494553] [encoder]: [NitrosNode] Loading extensions
[component_container-1] [INFO] [1678834616.454432413] [encoder]: [NitrosContext] Loading extension: gxf/std/libgxf_std.so
[component_container-1] [INFO] [1678834616.459611974] [encoder]: [NitrosContext] Loading extension: gxf/multimedia/libgxf_multimedia.so
[component_container-1] [INFO] [1678834616.461606314] [encoder]: [NitrosContext] Loading extension: gxf/cuda/libgxf_cuda.so
[component_container-1] [INFO] [1678834616.464939023] [encoder]: [NitrosContext] Loading extension: gxf/serialization/libgxf_serialization.so
[component_container-1] [INFO] [1678834616.468801046] [encoder]: [NitrosContext] Loading extension: gxf/tensorops/libgxf_tensorops.so
[component_container-1] [INFO] [1678834616.485286547] [encoder]: [NitrosContext] Loading extension: gxf/libgxf_encoder_extension.so
[component_container-1] [INFO] [1678834616.821149542] [encoder]: [NitrosNode] Loading graph to the optimizer
[component_container-1] [INFO] [1678834616.824797037] [encoder]: [NitrosNode] Running optimization
[component_container-1] [INFO] [1678834616.837014211] [encoder]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container-1] [INFO] [1678834616.838326725] [encoder]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container-1] [INFO] [1678834616.857919656] [encoder]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/encoder' in container '/encoder_container'
[component_container-1] [INFO] [1678834617.860907032] [encoder]: [NitrosNode] Starting post negotiation setup
[component_container-1] [INFO] [1678834617.861401305] [encoder]: [NitrosNode] Getting data format negotiation results
[component_container-1] [INFO] [1678834617.861518937] [encoder]: [NitrosSubscriber] Negotiation failed
[component_container-1] [INFO] [1678834617.861583705] [encoder]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/image_raw", data_format="nitros_image_bgr8"
[component_container-1] [INFO] [1678834617.861937114] [encoder]: [NitrosPublisher] Negotiation failed
[component_container-1] [INFO] [1678834617.862019354] [encoder]: [NitrosPublisher] Use only the compatible publisher: topic_name="/image_compressed", data_format="nitros_compressed_image"
[component_container-1] [INFO] [1678834617.862092058] [encoder]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container-1] [INFO] [1678834617.875232786] [encoder]: [NitrosNode] Wrote the final top level YAML graph to "/install/isaac_ros_h264_encoder/share/isaac_ros_h264_encoder/LDFRDFWRWF.yaml"
[component_container-1] [INFO] [1678834617.875436594] [encoder]: [NitrosNode] Calling user's pre-load-graph callback
[component_container-1] [INFO] [1678834617.875492434] [encoder]: [EncoderNode] preLoadGraphCallback().
[component_container-1] [INFO] [1678834617.875526098] [encoder]: [NitrosNode] Loading application
[component_container-1] [INFO] [1678834617.875594770] [encoder]: [NitrosContext] Loading application: '/install/isaac_ros_h264_encoder/share/isaac_ros_h264_encoder/LDFRDFWRWF.yaml'
[component_container-1] [INFO] [1678834617.882339230] [encoder]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container-1] [INFO] [1678834617.882821183] [encoder]: [NitrosNode] Calling user's post-load-graph callback
[component_container-1] [INFO] [1678834617.882906111] [encoder]: [EncoderNode] postLoadGraphCallback().
[component_container-1] [INFO] [1678834617.883187360] [encoder]: [NitrosContext] Initializing applicaiton...
[component_container-1] 2023-03-14 15:56:58.251 ERROR extensions/tensor_ops/TensorStream.cpp@96: tensor stream creation failed.
[component_container-1] 2023-03-14 15:56:58.251 ERROR gxf/std/entity_warden.cpp@379: Failed to initialize component 00010 (stream)
[component_container-1] 2023-03-14 15:56:58.251 ERROR gxf/core/runtime.cpp@551: Could not initialize entity 'LDFRDFWRWF_global' (E8): GXF_FAILURE
[component_container-1] 2023-03-14 15:56:58.251 ERROR gxf/std/program.cpp@191: Failed to activate entity 00008 named LDFRDFWRWF_global: GXF_FAILURE
[component_container-1] 2023-03-14 15:56:58.251 ERROR gxf/std/program.cpp@193: Deactivating...
[component_container-1] 2023-03-14 15:56:58.256 ERROR gxf/core/runtime.cpp@1158: Graph activation failed with error: GXF_FAILURE
[component_container-1] [ERROR] [1678834618.256950998] [encoder]: [NitrosContext] GxfGraphActivate Error: GXF_FAILURE
[component_container-1] [ERROR] [1678834618.257120246] [encoder]: [NitrosNode] runGraphAsync Error: GXF_FAILURE
[component_container-1] terminate called after throwing an instance of 'std::runtime_error'
[component_container-1]   what():  [NitrosNode] runGraphAsync Error: GXF_FAILURE
jaiveersinghNV commented 1 year ago

Could you please try this again on the latest Isaac ROS DP3 Release?

Additionally, could you provide the following information:

raitzjm commented 11 months ago

I am having the same issue in AGX Orin and AGX Xavier devkits running JetPack 5.1.1. I already opened a ticket in the ros_isaac_image_pipeline repo: https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_image_pipeline/issues/38 since that is where the issue is happening, in isaac_ros_image_pipeline/isaac_ros_image_proc/gxf/tensorops/extensions/tensorops/components/TensorStrea m.cpp@104: