search

defunctzombie / jetson-nano-image-maker

Create sd-card ready jetson nano images using Docker and Github Actions
32 stars 29 forks source link

tegra lib folder wrong nvidia-tegra.conf ld.so path #9

Open mrcmunir opened 9 months ago

mrcmunir commented 9 months ago

As the installation title said, the image generation are wrong.

It's not /usr/lib/tegra Tegra folder it's in /usr/lib/aarch64-linux-gnu/tegra and /usr/lib/aarch64-linux-gnu/tegra-egl

Otherwise any call to X11 or wayland GUI doesn't work will fail.

defunctzombie commented 5 months ago

I don't follow the issue. Which image generation is wrong? Could you be more specific on what you are referring to?

mrcmunir commented 5 months ago

I don't follow the issue. Which image generation is wrong? Could you be more specific on what you are referring to?

I mean that by default I don't see the symbolik link config files to tegra blobs folders How to see in Official Nvidia images.in etc/ld.so.conf.d/ The tegra libraries are copy in /usr/lib/aarch64-linux-gnu/tegra /usr/lib/aarch64-linux-gnu/tegra-egl But doesn't exist aarch64-linux-gnu_EGL.conf and aarch64-linux-gnu_GL.conf pointer to this in ld.so.conf files .

defunctzombie commented 5 months ago

Sorry but I don't understand what you are describing. There's nothing about those symlinks in this repo - maybe those are something that the tegra install itself does?

mrcmunir commented 5 months ago

Sorry but I don't understand what you are describing. There's nothing about those symlinks in this repo - maybe those are something that the tegra install itself does?

Yes It is related to the tegra packages that are installed Because if you look at the file inside in /etc/ld.so.conf.d nvidia-tegra.conf Points to /usr/lib/tegra and not /usr/lib/aarch64-linux-gnu/tegra

So you can see what I mean here is an example. https://github.com/balena-io-examples/jetson-examples/blob/master/jetson-nano/Dockerfile#L25

Since it is a folder that is not /usr/lib by default, it is necessary for ld.so.conf to find it by adding it.

defunctzombie commented 5 months ago

Can you find a reference to the nvidia installation docs about this file needing to be added? That above dockerfile does not have any comment on why this would be needed - in typical debian package installs such things are handled by the packages themselves.

What fails because you don't have this added reference in ld.so.conf?

I am happy to review a PR adding this entry but we'd need to write a comment or reference some docks on why we have to take this extra manual step.

mrcmunir commented 5 months ago

Can you find a reference to the nvidia installation docs about this file needing to be added? That above dockerfile does not have any comment on why this would be needed - in typical debian package installs such things are handled by the packages themselves.

You can guide yourself from the official images, you will see in that folder how they point to that folder The reasons are why I will explain below.

What fails because you don't have this added reference in ld.so.conf? I am happy to review a PR adding this entry but we'd need to write a comment or reference some docks on why we have to take this extra manual step.

The problem is the default path in the standard Linux structure is /usr/lib or /usr/local/lib but in this case is not for tegra folder.

If it does not have a .conf file as a pointer to a non-standardized folder such as the tegra/tegra-egl folder init, it will not find that library, causing an error when loading and you will not be able to access the GPU correctly. You can see when you do a dmesg that it doesn't find the library in the right place (i.e GLX init , GL core init , Vukan init and more ).

It looks like Nvidia customized the process manually and it was overlooked in their .deb packages.

It is a bit chaotic to understand in the official documentation since apparently Nvidia uses various scripts to generate the image https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/manifest_jetson_nano.html Since the structure has different scripts for each thing. apply_binaries.sh nv-apply-debs.sh