Add nvidia jailmaker config

[dasunsrule32]

This config will:

• Install all repos needed
• Install all dependancies
• Install docker
• Install nvidia-container-toolkit
• Reconfigure docker dameon to en- able the nvidia runtime

[neoKushan]

Rather than adding a whole separate config for nvidia, is there no way we can leverage the existing gpu_passthrough_nvidia flag to perform the nvidia specific installation and configuration?

Would probably require changes to the main jlmkr code, which I appreciate might not be ideal.

[dasunsrule32]

Maybe wrap the nvidia specific configs with if logic based on the value of gpu_passthrough_nvidia?

[neoKushan]

Yeah exactly, then there's only one template to maintain going forward rather than two

[Lockszmith-GH]

I always feel a bit queasy of placing too much logic in the initial_setup step, makes debugging rather hard, especially when getting into new-line, quoting and other possible issues. Would placing a script in the rootfs/ not be a better design?

[Jip-Hop]

Ideally we would not have a initial_setup at all. But that means setting up infrastructure to regularly build rootfs images from various distros with some packages installed and configured.

What are the quoting and line break issues you mention? The only requirement is that the script lines are indented...

Scripts can't be placed in the rootfs because then the config file would no longer be 'complete' on its own. The point of the config file is to be able to setup in a reproducible way without depending on external files.

[Lockszmith-GH]

Maybe just my bad luck, and indeed in my previous attempt there was a blank line without indentation. If the only limitation is indentation, I would agree, initial_setup might be the way to go.

[Jip-Hop]

Hi @dasunsrule32 please try the pre-release of jailmaker version v1.4.0 from the develop branch together with the updated docker config template. It now includes conditional install of nvidia-container-toolkit depending on the value of gpu_passthrough_nvidia. This means there's only 1 docker config template to maintain as @neoKushan suggested.

Please let me know if it works as intended so we can close this pull request.

[dasunsrule32]

Hi @dasunsrule32 please try the pre-release of jailmaker version v1.4.0 from the develop branch together with the updated docker config template. It now includes conditional install of nvidia-container-toolkit depending on the value of gpu_passthrough_nvidia. This means there's only 1 docker config template to maintain as @neoKushan suggested.

Please let me know if it works as intended so we can close this pull request.

Looks good! I built a machine from your config and only added the bind mounts needed for access to my data and enabled gpu_passthrough_nvidia. Launched Plex and watch live TV. It was using hardware transcoding. Didn't need to bind the /usr mount either for the nvidia libs.

diff --git a/templates/docker/config b/templates/docker/config
index 51f7f1d..5fb7655 100644
--- a/templates/docker/config
+++ b/templates/docker/config
@@ -1,6 +1,6 @@
 startup=0
 gpu_passthrough_intel=0
-gpu_passthrough_nvidia=0
+gpu_passthrough_nvidia=1
 # Turning off seccomp filtering improves performance at the expense of security
 seccomp=1

@@ -9,9 +9,14 @@ seccomp=1
 # Alternatively use --network-macvlan=eno1 instead of --network-bridge
 # Ensure to change eno1/br1 to the interface name you want to use
 # You may want to add additional options here, e.g. bind mounts
-systemd_nspawn_user_args=--network-bridge=br1
+systemd_nspawn_user_args=--network-bridge=br0
     --resolv-conf=bind-host
     --system-call-filter='add_key keyctl bpf'
+    --bind='/mnt/tank/containers/:/mnt/containers'
+    --bind='/mnt/tank/data/apps/:/mnt/data'
+    --bind='/mnt/tank/data/db/:/mnt/db'
+    --bind='/mnt/tank/media/:/mnt/media'
+    --bind='/mnt/tank/data/stacks:/opt/stacks'

[dasunsrule32]

Closing, as @Jip-Hop added to the origin/develop branch needed changes for nvidia to work out of the box now.

[Jip-Hop]

Nice!

Didn't need to bind the /usr mount either for the nvidia libs.

Which version of SCALE did you try this on? You're saying you don't experience https://github.com/Jip-Hop/jailmaker/issues/127?

[dasunsrule32]

Nice!

Didn't need to bind the /usr mount either for the nvidia libs.

Which version of SCALE did you try this on? You're saying you don't experience https://github.com/Jip-Hop/jailmaker/issues/127?

It's working fine on Dragonfish 24.04 with the updated config.