Recently, after running setup_dev_env.sh and installing NVIDIA libraries, there's an issue where part of NVML (nvml.h) is not installed. This affects the gpu_monitor node in system_monitor, which uses NVML. The gpu_monitor recognized NVML doesn't exist and publish errors as it is unable to access the GPU.
❯ find /usr -type f -name nvml.h
/usr/include/hwloc/nvml.h
Run setup_dev_env.sh and installi NVIDIA libraries.
❯ ./setup-dev-env.sh
...
[Warning] Some Autoware components depend on the CUDA, cuDNN and TensorRT NVIDIA libraries which have end-user license agreements that should be reviewed before installation.
Install NVIDIA libraries? [y/N]: y
Confirm that the NVIDIA's nvml.h is installed
❯ find /usr -type f -name nvml.h
/usr/local/cuda-12.3/targets/x86_64-linux/include/nvml.h
/usr/include/hwloc/nvml.h
Delete the build and install directories for system_monitor.
Description
Recently, after running
setup_dev_env.sh
and installing NVIDIA libraries, there's an issue where part of NVML (nvml.h
) is not installed. This affects thegpu_monitor
node insystem_monitor
, which uses NVML. Thegpu_monitor
recognized NVML doesn't exist and publish errors as it is unable to access the GPU.See also https://github.com/autowarefoundation/autoware.universe/issues/6787.
I'd like to explicitly install NVML as a workaround for this issue.
Tests performed
Completely remove NVIDIA drivers and libraries.
Confirm that only
hwloc/nvml.h
exists.Run
setup_dev_env.sh
and installi NVIDIA libraries.Confirm that the NVIDIA's
nvml.h
is installedDelete the build and install directories for system_monitor.
Build
system_monito
r and ensure build uses NVML (GPU PLATFORM: nvml
), and build completes successfully.Run Autoware.
Run runtime_monitor and Confirm the
gpu_monitor
does not report an error.Effects on system behavior
Not applicable.
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.
In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.
Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.
After all checkboxes are checked, anyone who has write access can merge the PR.