Add support for 390.x driver and refactor existing code

[tofurky]

This adds support for 390.x which required significant changes due to using nvmlInitWithFlags(unsigned int flags), whereas the previous nvmlInit() functions did not require passing any parameters. Effort was made to keep backwards compatibility (e.g. with 331 and below) by using a large amount of #if/#elif throughout nvml_fix.c. However, the old versions have not been tested beyond verifying they compile.

Also, the Makefile now checks if a supported TARGET_VERSION was provided before attempting any compilation. Without this change, the #error from nvml_fix.c is lost in a wall of other errors due to the refactoring. The make target is named "check_supported" and is called by default.

For the time being, only the offsets for x86_64 have been added.

Other changes:

• ${CC} is now called with "-s" which strips the library during the compilation step

• ${CC} is now called with "-Wl,--no-as-needed", which is required on newer gcc/binutils versions due to "--as-needed" being the new default

• the DESTDIR and prefix variables were removed in favor of a single libdir variable for clarity

• instead of bundling the nvml.h header from 390.x, it is now pulled in from the default system-wide header include path

Signed-off-by: Matt Merhar mattmerhar@protonmail.com

[CFSworks]

Thanks!!

[Thaodan]

Has Nvidia a pkgconfig file? This would avoid setting libdir static. Currently libdir is very Debian centric.

[tofurky]

no, there's no pkgconfig for nvidia that i found. i did try several things to autodetect the libdir (e.g. "find" and some other shell string manipulation), but it was all convoluted, potentially slow, and still wouldn't be 100% effective, so the compromise was to consolidate into a single variable and just add a note describing what to change it to. i doubt many people are building like 'make DESTDIR=/ PREFIX=... libdir=... TARGET_VER=...', and if they are, i'd think they're able to edit the Makefile. if there's a clean/fast/foolproof automated way to detect libdir then i agree it would be nice to have.

[tofurky]

though, come to think of it, README.md should be modified to reflect the changes i.e. libdir.

i'll do another PR to fix that.

[tofurky]

please take a look at the updated README at https://github.com/tofurky/nvml_fix/tree/libdir-doc-fix i'll open another PR, let me know if you think anything else should be changed.

[Thaodan]

Maybe libglvnd could help on this. I mean it should at least use the same libdir.

About the version thing there's a function in nvnml to query the driver version maybe that could help too.

[tofurky]

i think the documentation PR i submitted makes the compilation/install steps a lot more clear, but if we later decided to automate it:

maybe something based on dirname $(ldconfig -NXp|fgrep libnvidia-ml.so.1|awk '{print $4}') would work to find the libdir. and something like ldconfig -NXp|fgrep libnvidia-glcore.so.|awk '{print $1}'|cut -d '.' -f3-4 to find TARGET_VER.

still there are a few things to consider:

• it won't work on musl-based distros (e.g. alpine) due to the lack of proper ldconfig (but neither will the nvidia driver i guess, due to musl libc?)
• it wouldn't work on freebsd (there's a proprietary driver for it, offsets might differ anyways)
• there's no easy way to test it across all distros and different methods of installing nvidia driver like the official NVIDIA shell script installer and various distribution's packages
• making the shell commands portable against non-gnu awk/grep/etc would require some testing/simplification. i'm thinking in terms of those being busybox applets rather than the GNU versions.

logic could be like:

• if libdir/TARGET_VER is provided, use that
• if libdir/TARGET_VER isn't provided, try some ldconfig shell hackery
• if libdir/TARGET_VER is still unset, throw an $(error)

this would complicate the ability to compile the shim without having the driver/library installed, which is mentioned in the current README.md "Note: The nVidia drivers are not a dependency for building the shims."