search

Syllo / nvtop

GPU & Accelerator process monitoring for AMD, Apple, Huawei, Intel, NVIDIA and Qualcomm
Other
8.06k stars 292 forks source link

"No GPU to monitor" on Jetson AGX Xavier #150

Closed Jason-wwww closed 2 years ago

Jason-wwww commented 2 years ago

I have installed the 'nvtop' on Jetson AGX Xavier and I got this output "No GPU to monitor" while running "nvtop" to start it.

cxed commented 2 years ago

I have the exact same issue. Note that deviceQuery from cuda-samples works perfectly fine. How can we do whatever configuration is necessary to get nvtop working?

cxed commented 2 years ago

Jetpack's jtop also works.

Jason-wwww commented 2 years ago

Jetpack's jtop also works.

I can also use 'jtop', but 'nvtop' did not work.

Syllo commented 2 years ago

Hello, nvtop will try to dynamically load libnvidia-ml.so or libnvidia-ml.so.1 to query the GPU state. Is this library installed? Can you also please provide the nvidia driver version

cxed commented 2 years ago

I'm trying to figure out how to install libnvidia-ml.so on a Jetson Xavier. This is a SoC which needs the tegra drivers (present and working). There is no nvidia kernel module running and I don't know if that is necessary or valid. The display works, some diagnostic tools certainly find and can monitor the "GPU" (which isn't a proper graphics card in the classic sense). Any thoughts on how this critical library (libnvidia-ml) can be installed directly for aarch64?

Syllo commented 2 years ago

It usually comes packaged with the NVIDIA drivers or CUDA. https://packages.ubuntu.com/jammy/libnvidia-ml-dev seems to provide it for Ubuntu.

cxed commented 2 years ago

Thanks for your valuable response Syllo. Your link highlights the problem I think. The libnvidia-ml-dev packages for 22.04 does seem available for arm64, which is fine. But for 20.04 it does not seem to be available for that architecture. This is pretty frustrating since it's an Nvidia machine where Nvidia picks out everything - including what version of Ubuntu to run. Maybe not your problem and I do appreciate the good tip of what to look for. Any ideas for getting this on the stock distro Nvidia's own Jetpack installed (Ubuntu 20.04)? Thanks!

Syllo commented 2 years ago

According to the NVIDIA website it should be included with cuda. It may be in a non-standard location, you could search for it on your system find / -name libnvidia-ml.so.1.

There is always the hack of downloading the .deb package from debian, ignoring the missing dependencies, and remove it if it does not work.

Other than that I don't know.

Syllo commented 2 years ago

Closing for inactivity. Feel free to reopen if the issue persists.

pktiuk commented 1 year ago

This issue also exists on Nvidia Jetson Orin. Please reopen.

ratsputin commented 1 year ago

I'm using a Jetson Orin NX. That library does exist at the following path: /usr/local/cuda-11.4/targets/aarch64-linux/lib/stubs/libnvidia-ml.so That is, however, the only instance of that library.

As NVIDIA's jtop utility is horribly unstable (crashes as frequently as once every 2-3 minutes if processes come and go), I'll spend some time trying to get this to build on mine.

pktiuk commented 1 year ago

@ratsputin
Do you know how can I ensure nvtop uses this library? (due to operating system used on my jetson I have to use snap package)

Syllo commented 1 year ago

Hello, Nvtop uses the dynamic linker to load the libraries. If the libraries are not in the linker search path you can add additional paths using LD_LIBRARY_PATH (man ld.so), for example, LD_LIBRARY_PATH=/usr/local/cuda-11.4/targets/aarch64-linux/lib/stubs/ nvtop (this option is ignored in secure-execution mode).