Update deprecated TensorRT 8.5 codes for ROS2 humble

[xmfcx]

Checklist

• [X] I've read the contribution guidelines.
• [X] I've searched other issues and no duplicate issues were found.
• [X] I've agreed with the maintainers that I can plan this task.

Description

Currently in CI for humble & Ubuntu 22.04, we are using old (Ubuntu 20.04) packages for TensorRT.

In the latest (currently tensorrt-dev_8.5.1.7-1+cuda11.8) TensorRT version, some old APIs are deprecated.

Purpose

This issue is created to track upgrading the following 4 packages to compile and run Autoware Universe successfully.

Possible approaches

• [x] Wait until we deprecate galactic
• [ ] Update the CI and ansible to install latest version of TensorRT packages
• [x] Update traffic_light_classifier https://github.com/autowarefoundation/autoware.universe/pull/2325
• [x] Update traffic_light_ssd_fine_detector https://github.com/autowarefoundation/autoware.universe/pull/2326
• [x] Update tensorrt_yolo https://github.com/autowarefoundation/autoware.universe/pull/2327
• [x] Update lidar_centerpoint https://github.com/autowarefoundation/autoware.universe/pull/2328
• [ ] Update tensorrt_yolox (no PR created)

Definition of done

• [x] Make sure all TensorRT packages compile
• [x] Make sure traffic_light_classifier works normally
• [ ] Make sure traffic_light_ssd_fine_detector works normally
• [ ] Make sure tensorrt_yolo works normally
• [ ] Make sure lidar_centerpoint works normally
• [ ] Make sure tensorrt_yolox works normally
• [ ] CI is updated is updated to install the latest TensorRT packages
• [ ] Ansible code is updated to install the latest TensorRT packages
• [ ] PRs are merged

[xmfcx]

The ARM64 network repository https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/sbsa/ still doesn't have tensorrt-dev available.

I think we should also wait until this is available too.

Because the new code is not backwards compatible. (Update on 2022-12-21: I've updated them to be backwards compatible.)

We could also install old packages to ARM64 and new packages to x86_64 too but it would make things complicated.

What do you think, @wep21 @kenji-miyake ?

[kenji-miyake]

I think we should also wait until this is available too.

:+1: Using different versions for arm64 and amd64 would be too complicated.

[yukke42]

Nice! Doesn't lidar_apollo_instance_segmentation need to be updated?

[xmfcx]

@yukke42

Doesn't lidar_apollo_instance_segmentation need to be updated?

With all current PRs applied, everything compiled fine. I also checked lidar_apollo_instance_segmentation/lib/src/TrtNet.cpp and it doesn't look like it's using outdated APIs similar to ones I've changed.

Tomorrow I will try to compile it again just to be sure.

[xmfcx]

Update

• traffic_light_classifier fix was small, it is merged with https://github.com/autowarefoundation/autoware.universe/pull/2325 to support both versions.
• I've updated the PRs to be compatible with both versions. But as I was updating the code, I got stuck in updating enqueueV2 to enqueueV3 and updated versions doesn't work.
• Temporarily I've created https://github.com/autowarefoundation/autoware.universe/pull/2559 to temporarily have everything working.
• But I hope someone more experienced on TensorRT can help or take over this task.
• As I was working on this, https://github.com/autowarefoundation/autoware.universe/pull/2370 tensorrt_yolox package was added.
  • It needs to be updated too. I've added it to the list.

Actions

• For now with the https://github.com/autowarefoundation/autoware.universe/pull/2559 it works and I will take a break from this task.
• I am waiting for help on the existing open PRs.

cc @wep21 @kenji-miyake @yukkysaito

[xmfcx]

As for guidance in understanding the the enqueueV2 to enqueueV3 transition, following resources can be helpful:

• EnqueueV2 documentation
• EnqueueV3 documentation
• Runtime phase general helper doc: https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#runtime-phase
• https://github.com/NVIDIA/TensorRT/blob/main/samples/common/sampleInference.cpp the only place on the internet I could find where enqueueV3 is used.
• The sampleInference.cpp gets updated in this commit from v2 to v3 (search for sampleInference and expand) https://github.com/NVIDIA/TensorRT/commit/1d6bf36034bb07fc88d924ebf5d34d358298e545#diff-aae446aa5c914afd563008477f89fafa3cb0f9b58fb4d23b8aeeafb2ba4c693b

Also from my experience on the task:

• I didn't know how to incorporate the buffers from the v2 so I removed them, I think they should be used somehow.
  • My diff on how I did it in lidar_centerpoint
  • Probably bool setTensorAddress(char const* tensorName, void* data) noexcept must be used for this but I couldn't figure how.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity.