problem getting docker jailmaker to work

In my first attempt, i managed to get jailmaker running for initial setup

first i setup the networking first following the youtube guide. All done no issue.

*create the jailmaker dataset first

then go shell run this

cd /mnt/mypool/jailmaker
curl --location --remote-name https://raw.githubusercontent.com/Jip-Hop/jailmaker/main/jlmkr.py
chmod +x jlmkr.py
./jlmkr.py install

then setup the the ./jlmkr.py startup command in truenas advance.

at this point use jailmaker to run the docker script to set that up.

went to jailmaker shell to use docker commands like docker version which worked.

but i couldn't get the bind mounts to work. so while i was trying to fix that in the config, i noticed truenas update.

So i updated. Noticed jailmaker broke so re-ran the

./jlmkr.py install

some error about can't create symlinks or something

Something didn't work at this point, forgot what. Couldn't access the datasets via smb i think.

So i wanted to start from scratch, so i deleted the datasets, but then it complained a service uses it, but i deleted it anyway. I then rebooted truenas.

Came back tried to setup again.

When i tried to install jailmaker it says it's installed already.

I can run jailmaker commands so that part probably works.

I then tried to run the docker script to install that, the script runs, but then it encounters an error

jlmkr restart docker
Hit:1 http://deb.debian.org/debian bookworm InRelease
Hit:2 http://deb.debian.org/debian bookworm-updates InRelease
Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
Hit:4 https://download.docker.com/linux/debian bookworm InRelease
Reading package lists... Done                        
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ca-certificates is already the newest version (20230311).
curl is already the newest version (7.88.1-10+deb12u5).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Hit:1 http://deb.debian.org/debian bookworm InRelease
Hit:2 http://deb.debian.org/debian bookworm-updates InRelease
Hit:3 https://download.docker.com/linux/debian bookworm InRelease
Hit:4 http://deb.debian.org/debian-security bookworm-security InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
docker-ce is already the newest version (5:26.1.0-1~debian.12~bookworm).
docker-ce-cli is already the newest version (5:26.1.0-1~debian.12~bookworm).
containerd.io is already the newest version (1.6.31-1).
docker-buildx-plugin is already the newest version (0.14.0-1~debian.12~bookworm).
docker-compose-plugin is already the newest version (2.26.1-1~debian.12~bookworm).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Skipped mounting /dev/nvidia-modeset, it doesn't exist on the host...

Starting jail docker with the following command:

systemd-run --property=KillMode=mixed --property=Type=notify --property=RestartForceExitStatus=133 --property=SuccessExitStatus=133 --property=Delegate=yes --property=TasksMax=infinity --collect --setenv=SYSTEMD_NSPAWN_LOCK=0 --unit=jlmkr-docker --working-directory=./jails/docker '--description=My nspawn jail docker [created with jailmaker]' --property=ExecStartPre=/mnt/xxxxx/jailmaker/jails/docker/.ExecStartPre -- systemd-nspawn --keep-unit --quiet --boot --bind-ro=/sys/module --inaccessible=/sys/module/apparmor --machine=docker --directory=rootfs --bind=/dev/nvidia-uvm --bind=/dev/nvidiactl --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ptxjitcompiler.so.545.23.08 --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvcuvid.so.545.23.08 --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libcuda.so.545.23.08 --bind-ro=/usr/bin/nvidia-smi --bind-ro=/usr/lib/nvidia/current/nvidia-smi --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-encode.so.545.23.08 --bind=/dev/nvidia0 --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so.545.23.08 --bind-ro=/usr/bin/nvidia-persistenced --bind=/dev/nvidia-uvm-tools --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-cfg.so.545.23.08 --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-nvvm.so.545.23.08 --network-bridge=br1 --resolv-conf=bind-host '--system-call-filter=add_key keyctl bpf' --bind-ro=/mnt/xxxxx/Storage:/mnt/Storage --bind-ro=/mnt/xxxxx/Storage2:/mnt/Storage2 --bind=/mnt/xxxxx/jailmaker/docker/:/mnt/docker --bind=/mnt/xxxxx/docker/data/:/mnt/data --bind=/mnt/xxxxx/docker/compose/:/mnt/compose

Job for jlmkr-docker.service failed.
See "systemctl status jlmkr-docker.service" and "journalctl -xeu jlmkr-docker.service" for details.

Failed to start jail docker...
In case of a config error, you may fix it with:
jlmkr edit docker

root@xxxxx[/mnt/xxxxx/jailmaker]#

Apr 24 xxxxxsystemd[1]: Starting jlmkr-docker.service - My nspawn jail docker [created with jailmaker]…
░░ Subject: A start job for unit jlmkr-docker.service has begun execution
░░ Defined-By: systemd
░░ Support: [Debian -- User Support](https://www.debian.org/support)
░░
░░ A start job for unit jlmkr-docker.service has begun execution.
░░
░░ The job identifier is 2775.
Apr 24 xxxxx .ExecStartPre[386283]: PRE_START_HOOK
Apr 24 xxxxx systemd-nspawn[386286]: Failed to stat /mnt/xxxxx/docker/compose/: No such file or directory
Apr 24 xxxxx systemd[1]: jlmkr-docker.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: [Debian -- User Support](https://www.debian.org/support)
░░
░░ An ExecStart= process belonging to unit jlmkr-docker.service has exited.
░░
░░ The process’ exit code is ‘exited’ and its exit status is 1.
Apr 24 xxxxx systemd[1]: jlmkr-docker.service: Failed with result ‘exit-code’.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: [Debian -- User Support](https://www.debian.org/support)
░░
░░ The unit jlmkr-docker.service has entered the ‘failed’ state with result ‘exit-code’.
Apr 24 xxxxx xxxxx systemd[1]: Failed to start jlmkr-docker.service - My nspawn jail docker [created with jailmaker].
░░ Subject: A start job for unit jlmkr-docker.service has failed
░░ Defined-By: systemd
░░ Support: [Debian -- User Support](https://www.debian.org/support)
░░
░░ A start job for unit jlmkr-docker.service has finished with a failure.
░░
░░ The job identifier is 2775 and the job result is failed.
root@xxxxx[/mnt/xxxxx/jailmaker]#

any ideas?

Don't do this: --bind='/mnt/xxxxxx/jailmaker/docker/:/docker'

The jailmaker directory and the files inside it should be exclusively managed by jailmaker. As a rule of thumb, don't add any data there and don't mount any directories from this location inside the jail.

My advice is to remove jailmaker and start over with the install and creation of jails. Perhaps first create a very simple jail with jlmkr create test and build it out further. And it's not required to follow docs/basicconfig.md.

When you install docker using the config template, you don't have to do anything else (except for mounting some directories to make them available inside the jail, but these directories should NOT be children of the jailmaker directory).

Good luck!

Ty for the advice, i will try that :}

cd /mnt/MogVM/jailmaker/
curl --location --remote-name https://raw.githubusercontent.com/Jip-Hop/jailmaker/main/jlmkr.py
chmod +x jlmkr.py
./jlmkr.py install
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 70258  100 70258    0     0   160k      0 --:--:-- --:--:-- --:--:--  161k
systemd-nspawn is already installed.
Cannot create symlink because /usr/local/sbin/jlmkr is on a readonly filesystem.
The bash alias jlmkr is already present.
The zsh alias jlmkr is already present.
Done installing jailmaker.

Starting jail docker with the following command:

systemd-run --property=KillMode=mixed --property=Type=notify --property=RestartForceExitStatus=133 --property=SuccessExitStatus=133 --property=Delegate=yes --property=TasksMax=infinity --collect --setenv=SYSTEMD_NSPAWN_LOCK=0 --unit=jlmkr-docker --working-directory=./jails/docker '--description=My nspawn jail docker [created with jailmaker]' --property=ExecStartPre=/mnt/xxxxx/jailmaker/jails/docker/.ExecStartPre -- systemd-nspawn --keep-unit --quiet --boot --bind-ro=/sys/module --inaccessible=/sys/module/apparmor --machine=docker --directory=rootfs --bind=/dev/nvidiactl --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-encode.so.545.23.08 --bind-ro=/usr/lib/nvidia/current/nvidia-smi --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ptxjitcompiler.so.545.23.08 --bind-ro=/usr/bin/nvidia-smi --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvcuvid.so.545.23.08 --bind=/dev/nvidia-uvm-tools --bind=/dev/nvidia-uvm --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-nvvm.so.545.23.08 --bind=/dev/nvidia0 --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so.545.23.08 --bind-ro=/usr/bin/nvidia-persistenced --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libcuda.so.545.23.08 --bind-ro=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-cfg.so.545.23.08 --network-bridge=br1 --resolv-conf=bind-host '--system-call-filter=add_key keyctl bpf' --bind=/mnt/xxxxx/docker/:/mnt/docker --bind=/mnt/xxxxx/docker/data/:/mnt/data --bind=/mnt/xxxxx/docker/compose/:/mnt/compose --bind-ro=/mnt/xxxxx/Storage:/mnt/Storage --bind-ro=/mnt/xxxxx/Storage2:/mnt/Storage2

Job for jlmkr-docker.service failed.
See "systemctl status jlmkr-docker.service" and "journalctl -xeu jlmkr-docker.service" for details.

Failed to start jail docker...
In case of a config error, you may fix it with:
jlmkr edit docker

still not working after removing the child bind mount you mentioned.

i'll try basic and see if that works x-x;

Docker Using the docker config template is recommended if you want to run docker inside the jail. You may of course manually install docker inside a jail. But keep in mind that you need to add --system-call-filter='add_key keyctl bpf'

so if not using the script, they ask you to create the docker manually, but... i don't know how that is why i was using the script ;(

yes i created the test jail, no complaints, but at this point what to do x-x; yes install docker manually but how? xd

https://docs.docker.com/engine/install/debian/#install-using-the-repository

this is guess?

jlmkr shell test
Failed to get shell PTY: No machine 'test' known
root@Axxxxx[~]# jlmkr list      
NAME RUNNING STARTUP GPU_INTEL GPU_NVIDIA OS     VERSION ADDRESSES
test False   False   False     False      debian 12      -

 jlmkr status test
Unit jlmkr-test.service could not be found.
root@xxxx[~]#

my conclussion, even when not deploying docker with script, even just the basic simple jail test doesn't even work. so something else is up though not sure what x-x;

I'll try deleting the jailmakerdataset again to reinstall jailmaker from scratch though not sure what would change since i've done that before multiple times ;(

jlmkr remove test

CAUTION: Type "test" to confirm jail deletion!

test

Cleaning up: jails/test.
Removing ZFS Dataset xxxx/jailmaker/jails/test

After creating the simple test jail it is not running as indicated by the output of jlmkr list.

You may try starting it with jlmkr start test.

Please read the main README carefully.

Ok so i tried to start from scratch. when trying to delete dataset it says it's in use.

so i reboot truenas.

then i delete dataset (the jailmaker dataset)

then i started over again following the guide (this covers setting up the dataset and also what commands to run in shell) https://www.truenas.com/docs/scale/scaletutorials/apps/sandboxes/

Configure a Dataset for Jailmaker

Log in to the web interface and go to Datasets.

Select your root pool and click Add Dataset:

a. Name the dataset jailmaker.

b. Leave all other settings at their defaults.

c. Click Save.

Open a Shell (SSH preferred) session and run these commands as root:

a. cd /mnt/tank/jailmaker/ . Replace tank with the name of your pool.

b. curl --location --remote-name https://raw.githubusercontent.com/Jip-Hop/jailmaker/main/jlmkr.py

c. chmod +x jlmkr.py

Run Jailmaker when System Starts

Before making any sandboxes, configure TrueNAS to run the Jailmaker tool when the system starts. This ensures the sandboxes start properly.

Log in to the web interface and go to System Settings > Advanced.

Find the Init/Shutdown Scripts widget and click Add:

a. Enter this or a similar note in Description: Jailmaker Startup

b. Set Type to Command.

c. Enter this string in Command: /mnt/tank/jailmaker/jlmkr.py startup . Replace tank with the name of your pool.

d. Set When to Post Init.

e. Set the Enabled checkbox.

f. Leave Timeout at the default and click Save. If you intend to create many sandboxes, increase the timeout integer to a longer wait period.

once done, i then try install but i still ran into the same issue.

jlmkr create test

jlmkr shell test   
Failed to get shell PTY: No machine 'test' known

jlmkr list
NAME RUNNING STARTUP GPU_INTEL GPU_NVIDIA OS     VERSION ADDRESSES
test False   False   False     False      debian 12      -

jlmkr status test
Unit jlmkr-test.service could not be found.

i followed all the steps according to the guides :{ not much else i can do.

Please read the main README carefully.

must have read it a hundred times already, but i'll do it again. but as you can see from all the steps i listed, i really did follow the instructions to the tee but no dice.

The thing is i did get it to work the first time. But i did explain that it broke down and never worked again at some point. so that is my situation. I guess doing a factory reset may work, but that is a bit extreme ^-^;

Think of the jail as a machine of it's own, when it's created it's just a box that has not been plugged to power yet. After you start it, it's powered on, that's when you can interact with it.

@mooglestiltzkin as you can see, after create, the list command shows that the test jail is NOT running (RUNNING is False).

before you can shell into it, you need to start it.

jlmkr start test

then you should be able to run status and shell on it.

When you see instructions to install something (for example docker), that means, you login into the machine (with jlmkr shell ... after it has been powerd on) and then you are inside a linux machine of its own, and you can install whatever you want in it.

BTW, the reason you were told the dataset was busy when you tried destroying it, it was likely because your jail was started, if you would have done a jlmkr stop ... you would have probable been able to delete the jail's dataset without issue. Better yet, jlmkr remove ... would have problable done it for you.

derp. you were right

jlmkr start test 

Starting jail test with the following command:

systemd-run --collect --property=Delegate=yes --property=RestartForceExitStatus=133 --property=SuccessExitStatus=133 --property=TasksMax=infinity --property=Type=notify --setenv=SYSTEMD_NSPAWN_LOCK=0 --property=KillMode=mixed --unit=jlmkr-test --working-directory=./jails/test '--description=My nspawn jail test [created with jailmaker]' -- systemd-nspawn --bind-ro=/sys/module --boot --inaccessible=/sys/module/apparmor --quiet --keep-unit --machine=test --directory=rootfs

Running as unit: jlmkr-test.service

jlmkr shell test
Connected to machine test. Press ^] three times within 1s to exit session.
root@test:~#

it works yay..... *faint X_X:

Now i need to lie down and rest. i've been at this for hours. i know pathetic but i'm a newb ;(

I will be sure to note this down so others don't make my mistake. i'm updating my project journal

https://forums.truenas.com/t/qnap-ts-877-truenas-journal/1646/1

the documentation is good, is just that people like me will still make mistakes unfortunately x-x; so i'll try narrate what to watch out for (in my experience setting this up)

*e.g.

Troubleshooting

did you remember to start the jail? type jlmkr start test Only after that will you be able to access the shell jlmkr shell test
did you create the jailmaker dataset correctly? refer to this and follow it exactly, no deviation https://www.truenas.com/docs/scale/scaletutorials/apps/sandboxes/
if configuring docker config, do not bind mount to the jail dataset. the bind mount only should be used for other datasets residing outside the jailmaker dataset only.
Try setup networking first before starting. If doing bridges follow this guide https://www.youtube.com/watch?v=7clQw132w58 and refer to the networking doc for more details on the specifics https://github.com/Jip-Hop/jailmaker/blob/main/NETWORKING.md

etc

@Jip-Hop I think this is a case where the order of items in the README.md might be tripping users without much experience. (Sorry @mooglestiltzkin to 'talk over you' here, no disrespect).

Line 69 states:

After answering some questions you should have your first jail up and running!

It is not running yet, it needs to be started.

Then it is followed by the Startup Jails on Boot section, and only then with the Start Jail section.

A little edit there, and I think those who take things much more literal, will have an easier time following an updated version of the instrucitons.

Maybe even introduce list before start, and explain what is being seen there with a link to systemd-nspawn's documentation about it.

I'll post a PR later if you can't get to it.

As a general troubleshooting question, do you have any apps running? Also, what are the specs of the machine? I am running on an N100 nas motherboard with 32gb of DDR5. I found that if k3s was running and my zfs cache was decently populated - jails would not work and I would get lots of errors. I had wanted to get jails working first before disabling my apps - but I found it was much easier to disable those first and then get jails running. Of course, could be due to my low power system.

Further - I know RTFM seems a bit dismissive, but as a fellow newb - every line is important. There is a link on the cover readme that takes you to a networking document - it was key for me to get networking to work. That document then links to a video about setting up a network bridge. The readme alone doesnt highlight that you need to set a network bridge FIRST in the webui and then use the name bridge you created as the argument for --network-bridge=(the name you gave your bridge).

A lot of good information is buried at the bottom of the readme and the other documentation - so take it all in.

so i thought now i can install the docker script and then start the docker jail, but it can't

jlmkr start docker

Job for jlmkr-docker.service failed.
See "systemctl status jlmkr-docker.service" and "journalctl -xeu jlmkr-docker.service" for details.

Failed to start jail docker...
In case of a config error, you may fix it with:
jlmkr edit docker

only the test jail worked. in the sense that i could at least start that jail and get into the jail's shell. hm

This is my setup

TS-877 reflashed to truenas dragonfish

AMD Ryzen 1400

16gb ddr4

boot drive is a m.2 nvme ssd on an external usb enclosure

4x 4tb hard drives seagate ironwolf

2x m.2 sata ssds (this is used for the vm pool. and where the jailmaker dataset resides)

new truenas setup barely a few days old. so i came from a clean fresh environment starting on the dragonfish RC1 then updated to the latest stable today.

No i don't have any apps other than truenas.

i did try the kubenetes k3, to test unifi controller, tested that works. then i opted to unlink the app pool and restart. Because i was planning to only use docker to then setup portainer to then be able to deploy my docker containers which i am familiar with.

so i deleted the ixsystems, app data sets, then also the jailmaker dataset (because i was gonna redo it from scratch). Then i got a warning some service uses one of those datasets, i just click delete anyway.

I think things went wrong with jailmaker from that point onward. At least before this, i could at least go into docker shell even if i did not yet figure out the bind mounts at that time and acls/permissions and stuff.

but right now i am in the process of trying to setup and get jailmaker to work so i can deploy docker, then to then docker compose up portainer, then to deploy the rest the docker containers. that is the plan.

do you have networking working from within the test jail?

can you ping the world?

yes, so i

jlmkr create test

jlmkr start test

jlmkr shell test

ping cnn.com

yes it's working

that said i don't know how to stop the pinging so i re-enter shell xd

jlmkr status test
● jlmkr-test.service - My nspawn jail test [created with jailmaker]
     Loaded: loaded (/run/systemd/transient/jlmkr-test.service; transient)
  Transient: yes
     Active: active (running) since Wed 2024-04-24 xxxxxxx; 2min 11s ago
   Main PID: 112394 (systemd-nspawn)
     Status: "Container running: Ready."
      Tasks: 7
     Memory: 28.9M
        CPU: 1.228s
     CGroup: /system.slice/jlmkr-test.service
             ├─payload
             │ ├─init.scope
             │ │ └─112396 /usr/lib/systemd/systemd
             │ └─system.slice
             │   ├─console-getty.service
             │   │ └─112464 /sbin/agetty -o "-p -- \\u" --noclear --keep-baud - xxxxxxx
             │   ├─dbus.service
             │   │ └─112461 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
             │   ├─systemd-journald.service
             │   │ └─112440 /lib/systemd/systemd-journald
             │   ├─systemd-logind.service
             │   │ └─112462 /lib/systemd/systemd-logind
             │   └─systemd-resolved.service
             │     └─112459 /lib/systemd/systemd-resolved
             └─supervisor
               └─112394 /usr/bin/systemd-nspawn --bind-ro=/sys/module --boot --inaccessible=/sys/module/apparmor --quiet --keep-unit --machine=test --directory=rootfs

Apr 24 xxxxxxx systemd-nspawn[112394]: [  OK  ] Started dbus.service - D-Bus System Message Bus.
Apr 24 xxxxxxx systemd-nspawn[112394]: [  OK  ] Started systemd-logind.service - User Login Management.
Apr 24 xxxxxxx systemd-nspawn[112394]: [  OK  ] Reached target multi-user.target - Multi-User System.
Apr 24 xxxxxxx systemd-nspawn[112394]: [  OK  ] Reached target graphical.target - Graphical Interface.
Apr 24 xxxxxxx systemd-nspawn[112394]:          Starting systemd-update-utmp-runlev… Record Runlevel Change in UTMP...
Apr 24 xxxxxxx systemd-nspawn[112394]: [  OK  ] Finished systemd-update-utmp-runlev… - Record Runlevel Change in UTMP.
Apr 24 xxxxxxxsystemd-nspawn[112394]: [  OK  ] Finished systemd-machine-id-commit.…it a transient machine-id on disk.
Apr 24 xxxxxxx systemd-nspawn[112394]: 
Apr 24 xxxxxxx systemd-nspawn[112394]: Debian GNU/Linux 12 test pts/0
Apr 24 xxxxxxx  systemd-nspawn[112394]:

that said i don't know how to stop the pinging so i re-enter shell xd

😄 fair enough - (you can limit the ping with -c 3, the shell is a finicky)

So when you said jlmkr start docker failed, what instructions did you follow to create the docker container? Did you use the template?

curl -sL "https://github.com/Jip-Hop/jailmaker/raw/main/templates/docker/config" -o ~/jlmkr-docker-config
vi ~/jlmkr-docker-config # edit the template with the settings that are suitable for you
jlmkr create --start --config ~/jlmkr-docker-config

Further - I know RTFM seems a bit dismissive, but as a fellow newb - every line is important. There is a link on the cover readme that takes you to a networking document - it was key for me to get networking to work. That document then links to a video about setting up a network bridge. The readme alone doesnt highlight that you need to set a network bridge FIRST in the webui and then use the name bridge you created as the argument for --network-bridge=(the name you gave your bridge).

you are absolutely right. That's why i think another manual for dummies is needed. What to watch out for ;)

So when you said jlmkr start docker failed, what instructions did you follow to create the docker container? Did you use the template?

Yes i used the template https://github.com/Jip-Hop/jailmaker/blob/main/templates/docker/README.md

jlmkr create --start --config /mnt/xxxxx/docker/config docker

With that template config, i then slowly modified e.g. added the networking bridge, then the bind mounts. Flagged nvidia graphics card to 1 and change startup to 1.

That was about it i think

this is the docker config i was trying to install

startup=1
gpu_passthrough_intel=0
gpu_passthrough_nvidia=1
# Turning off seccomp filtering improves performance at the expense of security
seccomp=1

# Use macvlan networking to provide an isolated network namespace,
# so docker can manage firewall rules
# Alternatively use --network-bridge=br1 instead of --network-macvlan
# Ensure to change eno1/br1 to the interface name you want to use
# You may want to add additional options here, e.g. bind mounts
systemd_nspawn_user_args=--network-bridge=br1
    --resolv-conf=bind-host
    --system-call-filter='add_key keyctl bpf'   
    --bind='/mnt/xxxxxx/docker/:/mnt/docker'
    --bind='/mnt/xxxxxx/docker/data/:/mnt/data'
    --bind='/mnt/xxxxxx/docker/compose/:/mnt/compose'
    --bind-ro='/mnt/xxxxxx2/Storage:/mnt/Storage'
    --bind-ro='/mnt/xxxxxx2/Storage2:/mnt/Storage2'

# Script to run on the HOST before starting the jail
# Load kernel module and config kernel settings required for docker
pre_start_hook=#!/usr/bin/bash
    set -euo pipefail
    echo 'PRE_START_HOOK'
    echo 1 > /proc/sys/net/ipv4/ip_forward
    modprobe br_netfilter
    echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
    echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables

# Only used while creating the jail
distro=debian
release=bookworm

# Install docker inside the jail:
# https://docs.docker.com/engine/install/debian/#install-using-the-repository
# NOTE: this script will run in the host networking namespace and ignores
# all systemd_nspawn_user_args such as bind mounts
initial_setup=#!/usr/bin/bash
    set -euo pipefail

    apt-get update && apt-get -y install ca-certificates curl
    install -m 0755 -d /etc/apt/keyrings
    curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
    chmod a+r /etc/apt/keyrings/docker.asc

    echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
    $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
    tee /etc/apt/sources.list.d/docker.list > /dev/null
    apt-get update
    apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# You generally will not need to change the options below
systemd_run_default_args=--property=KillMode=mixed
    --property=Type=notify
    --property=RestartForceExitStatus=133
    --property=SuccessExitStatus=133
    --property=Delegate=yes
    --property=TasksMax=infinity
    --collect
    --setenv=SYSTEMD_NSPAWN_LOCK=0

systemd_nspawn_default_args=--keep-unit
    --quiet
    --boot
    --bind-ro=/sys/module
    --inaccessible=/sys/module/apparmor

you are absolutely right. That's why i think another manual for dummies is needed. What to watch out for ;)

I think the edits I suggested above should be sufficient.

What's the output of:

sudo journalctl -eu jlmkr-docker.service

have you tried it with gpu_passthrough_nvidia=0?

i deleted docker jlmkr remove docker

then jlmkr create --start --config /mnt/MogVM/docker/config docker

same result.

then

root@xxxxx[~]# sudo journalctl -eu jlmkr-docker.service
~
~
~
~
~
~
~
~
~
~
~
Apr 24 xxxxxxxxx systemd[1]: Starting jlmkr-docker.service - My nspawn jail docker [created with jailmaker]...
Apr 24 xxxxxxxxx  .ExecStartPre[26059]: PRE_START_HOOK
Apr 24 xxxxxxxxx  systemd-nspawn[26062]: Failed to stat /mnt/xxxxxxxxx /docker/compose/: No such file or directory
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Main process exited, code=exited, status=1/FAILURE
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Failed with result 'exit-code'.
Apr 24 xxxxxxxxx  systemd[1]: Failed to start jlmkr-docker.service - My nspawn jail docker [created with jailmaker].
Apr 24 xxxxxxxxx  systemd[1]: Starting jlmkr-docker.service - My nspawn jail docker [created with jailmaker]...
Apr 24 xxxxxxxxx  .ExecStartPre[65193]: PRE_START_HOOK
Apr 24 xxxxxxxxx  systemd-nspawn[65196]: Failed to stat /mnt/xxxxxxxxx /docker/compose/: No such file or directory
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Main process exited, code=exited, status=1/FAILURE
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Failed with result 'exit-code'.
Apr 24 xxxxxxxxx  systemd[1]: Failed to start jlmkr-docker.service - My nspawn jail docker [created with jailmaker].
Apr 24 xxxxxxxxx  systemd[1]: Starting jlmkr-docker.service - My nspawn jail docker [created with jailmaker]...
Apr 24 xxxxxxxxx  .ExecStartPre[98763]: PRE_START_HOOK
Apr 24 xxxxxxxxx  systemd-nspawn[98766]: Failed to stat /mnt/xxxxxxxxx /docker/compose/: No such file or directory
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Main process exited, code=exited, status=1/FAILURE
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Failed with result 'exit-code'.
Apr 24 xxxxxxxxx  systemd[1]: Failed to start jlmkr-docker.service - My nspawn jail docker [created with jailmaker].
Apr 24 xxxxxxxxx  systemd[1]: Starting jlmkr-docker.service - My nspawn jail docker [created with jailmaker]...
Apr 24 xxxxxxxxx  .ExecStartPre[99830]: PRE_START_HOOK
Apr 24 xxxxxxxxx  systemd-nspawn[99833]: Failed to stat /mnt/xxxxxxxxx /docker/compose/: No such file or directory
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Main process exited, code=exited, status=1/FAILURE
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Failed with result 'exit-code'.
Apr 24 xxxxxxxxx  systemd[1]: Failed to start jlmkr-docker.service - My nspawn jail docker [created with jailmaker].
Apr 24 xxxxxxxxx  systemd[1]: Starting jlmkr-docker.service - My nspawn jail docker [created with jailmaker]...
Apr 24 xxxxxxxxx  .ExecStartPre[162545]: PRE_START_HOOK
Apr 24 xxxxxxxxx  systemd-nspawn[162552]: Failed to stat /mnt/xxxxxxxxx /docker/compose/: No such file or directory
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Main process exited, code=exited, status=1/FAILURE
Apr 24 xxxxxxxxx  systemd[1]: jlmkr-docker.service: Failed with result 'exit-code'.
Apr 24 xxxxxxxxx  systemd[1]: Failed to start jlmkr-docker.service - My nspawn jail docker [created with jailmaker].
root@xxxxxxxxx [~]#

i'll try change the graphic card setting you mentioned

not sure which of this solved the issue. but regardless it worked.

i do have a nvidia 1030 GTX MSI AERO 2gb vram low profile graphics card. it does work, because i can connect it to the monitor and it shows display when booting up truenas. fyi.

IT WORKED.

I did 2 things

change the graphics card for nvidia to 0
i noticed in the output it couldn't find the /mnt/xxxxxxxxx /docker/compose/

so i did make a /mnt/xxxxxxxxx /docker/data/ but i did not make a compose because i forgot or i also assumed it would simply create it. but guess not.

So this is what i did. as you can see, i removed the docker jail to redo it again. I listed all the steps and what happened exactly.

root@xxxxxxx[~]# jlmkr remove docker

CAUTION: Type "docker" to confirm jail deletion!

docker

Cleaning up: jails/docker.
Removing ZFS Dataset xxxxxxx/jailmaker/jails/docker
root@AINZ[~]# jlmkr create --start --config /mnt/xxxxxxx/docker/config docker
USE THIS SCRIPT AT YOUR OWN RISK!
IT COMES WITHOUT WARRANTY AND IS NOT SUPPORTED BY IXSYSTEMS.
Creating jail docker from config template /mnt/MogVM/docker/config.

TIP: Run `jlmkr create` without any arguments for interactive config.
Or use CLI args to override the default options.
For more info, run: `jlmkr create --help`

Creating ZFS Dataset xxxxxxx/jailmaker/jails/docker
Using image from local cache
Unpacking the rootfs

---
You just created a Debian bookworm amd64 (20240424_05:24) container.

To enable SSH, run: apt install openssh-server
No default root or user password are set by LXC.
Hit:1 http://deb.debian.org/debian bookworm InRelease
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
Fetched 55.4 kB in 1s (78.4 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libbrotli1 libcurl4 libldap-2.5-0 libldap-common libnghttp2-14 libpsl5 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 openssl publicsuffix
Suggested packages:
  libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap libsasl2-modules-otp libsasl2-modules-sql
The following NEW packages will be installed:
  ca-certificates curl libbrotli1 libcurl4 libldap-2.5-0 libldap-common libnghttp2-14 libpsl5 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 openssl publicsuffix
0 upgraded, 15 newly installed, 0 to remove and 0 not upgraded.
Need to get 3409 kB of archives.
After this operation, 7291 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main amd64 openssl amd64 3.0.11-1~deb12u2 [1419 kB]
Get:2 http://deb.debian.org/debian bookworm/main amd64 ca-certificates all 20230311 [153 kB]
Get:3 http://deb.debian.org/debian bookworm/main amd64 libbrotli1 amd64 1.0.9-2+b6 [275 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules-db amd64 2.1.28+dfsg-10 [20.3 kB]
Get:5 http://deb.debian.org/debian bookworm/main amd64 libsasl2-2 amd64 2.1.28+dfsg-10 [59.7 kB]
Get:6 http://deb.debian.org/debian bookworm/main amd64 libldap-2.5-0 amd64 2.5.13+dfsg-5 [183 kB]
Get:7 http://deb.debian.org/debian bookworm/main amd64 libnghttp2-14 amd64 1.52.0-1+deb12u1 [72.4 kB]
Get:8 http://deb.debian.org/debian bookworm/main amd64 libpsl5 amd64 0.21.2-1 [58.7 kB]
Get:9 http://deb.debian.org/debian bookworm/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2+b2 [60.8 kB]
Get:10 http://deb.debian.org/debian bookworm/main amd64 libssh2-1 amd64 1.10.0-3+b1 [179 kB]
Get:11 http://deb.debian.org/debian bookworm/main amd64 libcurl4 amd64 7.88.1-10+deb12u5 [390 kB]
Get:12 http://deb.debian.org/debian bookworm/main amd64 curl amd64 7.88.1-10+deb12u5 [315 kB]
Get:13 http://deb.debian.org/debian bookworm/main amd64 libldap-common all 2.5.13+dfsg-5 [29.3 kB]
Get:14 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules amd64 2.1.28+dfsg-10 [66.6 kB]
Get:15 http://deb.debian.org/debian bookworm/main amd64 publicsuffix all 20230209.2326-1 [126 kB]
Fetched 3409 kB in 0s (8868 kB/s)       
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package openssl.
(Reading database ... 13564 files and directories currently installed.)
Preparing to unpack .../00-openssl_3.0.11-1~deb12u2_amd64.deb ...
Unpacking openssl (3.0.11-1~deb12u2) ...
Selecting previously unselected package ca-certificates.
Preparing to unpack .../01-ca-certificates_20230311_all.deb ...
Unpacking ca-certificates (20230311) ...
Selecting previously unselected package libbrotli1:amd64.
Preparing to unpack .../02-libbrotli1_1.0.9-2+b6_amd64.deb ...
Unpacking libbrotli1:amd64 (1.0.9-2+b6) ...
Selecting previously unselected package libsasl2-modules-db:amd64.
Preparing to unpack .../03-libsasl2-modules-db_2.1.28+dfsg-10_amd64.deb ...
Unpacking libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ...
Selecting previously unselected package libsasl2-2:amd64.
Preparing to unpack .../04-libsasl2-2_2.1.28+dfsg-10_amd64.deb ...
Unpacking libsasl2-2:amd64 (2.1.28+dfsg-10) ...
Selecting previously unselected package libldap-2.5-0:amd64.
Preparing to unpack .../05-libldap-2.5-0_2.5.13+dfsg-5_amd64.deb ...
Unpacking libldap-2.5-0:amd64 (2.5.13+dfsg-5) ...
Selecting previously unselected package libnghttp2-14:amd64.
Preparing to unpack .../06-libnghttp2-14_1.52.0-1+deb12u1_amd64.deb ...
Unpacking libnghttp2-14:amd64 (1.52.0-1+deb12u1) ...
Selecting previously unselected package libpsl5:amd64.
Preparing to unpack .../07-libpsl5_0.21.2-1_amd64.deb ...
Unpacking libpsl5:amd64 (0.21.2-1) ...
Selecting previously unselected package librtmp1:amd64.
Preparing to unpack .../08-librtmp1_2.4+20151223.gitfa8646d.1-2+b2_amd64.deb ...
Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ...
Selecting previously unselected package libssh2-1:amd64.
Preparing to unpack .../09-libssh2-1_1.10.0-3+b1_amd64.deb ...
Unpacking libssh2-1:amd64 (1.10.0-3+b1) ...
Selecting previously unselected package libcurl4:amd64.
Preparing to unpack .../10-libcurl4_7.88.1-10+deb12u5_amd64.deb ...
Unpacking libcurl4:amd64 (7.88.1-10+deb12u5) ...
Selecting previously unselected package curl.
Preparing to unpack .../11-curl_7.88.1-10+deb12u5_amd64.deb ...
Unpacking curl (7.88.1-10+deb12u5) ...
Selecting previously unselected package libldap-common.
Preparing to unpack .../12-libldap-common_2.5.13+dfsg-5_all.deb ...
Unpacking libldap-common (2.5.13+dfsg-5) ...
Selecting previously unselected package libsasl2-modules:amd64.
Preparing to unpack .../13-libsasl2-modules_2.1.28+dfsg-10_amd64.deb ...
Unpacking libsasl2-modules:amd64 (2.1.28+dfsg-10) ...
Selecting previously unselected package publicsuffix.
Preparing to unpack .../14-publicsuffix_20230209.2326-1_all.deb ...
Unpacking publicsuffix (20230209.2326-1) ...
Setting up libpsl5:amd64 (0.21.2-1) ...
Setting up libbrotli1:amd64 (1.0.9-2+b6) ...
Setting up libsasl2-modules:amd64 (2.1.28+dfsg-10) ...
Setting up libnghttp2-14:amd64 (1.52.0-1+deb12u1) ...
Setting up libldap-common (2.5.13+dfsg-5) ...
Setting up libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ...
Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ...
Setting up libsasl2-2:amd64 (2.1.28+dfsg-10) ...
Setting up libssh2-1:amd64 (1.10.0-3+b1) ...
Setting up openssl (3.0.11-1~deb12u2) ...
Setting up publicsuffix (20230209.2326-1) ...
Setting up libldap-2.5-0:amd64 (2.5.13+dfsg-5) ...
Setting up ca-certificates (20230311) ...
Updating certificates in /etc/ssl/certs...
140 added, 0 removed; done.
Setting up libcurl4:amd64 (7.88.1-10+deb12u5) ...
Setting up curl (7.88.1-10+deb12u5) ...
Processing triggers for libc-bin (2.36-9+deb12u6) ...
Processing triggers for ca-certificates (20230311) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Hit:1 http://deb.debian.org/debian bookworm InRelease
Hit:2 http://deb.debian.org/debian bookworm-updates InRelease               
Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease     
Get:4 https://download.docker.com/linux/debian bookworm InRelease [43.3 kB]
Get:5 https://download.docker.com/linux/debian bookworm/stable amd64 Packages [21.6 kB]
Fetched 65.0 kB in 1s (124 kB/s)    
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  apparmor docker-ce-rootless-extras git git-man iptables less libcurl3-gnutls liberror-perl libglib2.0-0 libglib2.0-data libicu72 libip6tc2 libjansson4 libltdl7 libnetfilter-conntrack3 libnfnetlink0
  libnftables1 libnftnl11 libslirp0 libxml2 nftables patch pigz shared-mime-info slirp4netns xdg-user-dirs xz-utils
Suggested packages:
  apparmor-profiles-extra apparmor-utils aufs-tools cgroupfs-mount | cgroup-lite gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-email git-gui gitk gitweb git-cvs git-mediawiki git-svn
  firewalld kmod low-memory-monitor ed diffutils-doc
The following NEW packages will be installed:
  apparmor containerd.io docker-buildx-plugin docker-ce docker-ce-cli docker-ce-rootless-extras docker-compose-plugin git git-man iptables less libcurl3-gnutls liberror-perl libglib2.0-0 libglib2.0-data
  libicu72 libip6tc2 libjansson4 libltdl7 libnetfilter-conntrack3 libnfnetlink0 libnftables1 libnftnl11 libslirp0 libxml2 nftables patch pigz shared-mime-info slirp4netns xdg-user-dirs xz-utils
0 upgraded, 32 newly installed, 0 to remove and 0 not upgraded.
Need to get 147 MB of archives.
After this operation, 549 MB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main amd64 pigz amd64 2.6-1 [64.0 kB]
Get:2 https://download.docker.com/linux/debian bookworm/stable amd64 containerd.io amd64 1.6.31-1 [29.8 MB]
Get:3 http://deb.debian.org/debian bookworm/main amd64 less amd64 590-2 [131 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 libjansson4 amd64 2.14-2 [40.8 kB]
Get:5 http://deb.debian.org/debian bookworm/main amd64 libnftnl11 amd64 1.2.4-2 [61.6 kB]
Get:6 http://deb.debian.org/debian bookworm/main amd64 libnftables1 amd64 1.0.6-2+deb12u2 [299 kB]
Get:7 http://deb.debian.org/debian bookworm/main amd64 nftables amd64 1.0.6-2+deb12u2 [70.3 kB]
Get:8 http://deb.debian.org/debian bookworm/main amd64 xz-utils amd64 5.4.1-0.2 [471 kB]
Get:9 http://deb.debian.org/debian bookworm/main amd64 apparmor amd64 3.0.8-3 [616 kB]
Get:10 http://deb.debian.org/debian bookworm/main amd64 libip6tc2 amd64 1.8.9-2 [19.4 kB]
Get:11 http://deb.debian.org/debian bookworm/main amd64 libnfnetlink0 amd64 1.0.2-2 [15.1 kB]
Get:12 http://deb.debian.org/debian bookworm/main amd64 libnetfilter-conntrack3 amd64 1.0.9-3 [40.7 kB]
Get:13 http://deb.debian.org/debian bookworm/main amd64 iptables amd64 1.8.9-2 [360 kB]  
Get:14 http://deb.debian.org/debian bookworm/main amd64 libcurl3-gnutls amd64 7.88.1-10+deb12u5 [385 kB]
Get:15 http://deb.debian.org/debian bookworm/main amd64 liberror-perl all 0.17029-2 [29.0 kB]
Get:16 http://deb.debian.org/debian bookworm/main amd64 git-man all 1:2.39.2-1.1 [2049 kB]
Get:17 http://deb.debian.org/debian bookworm/main amd64 git amd64 1:2.39.2-1.1 [7171 kB]
Get:18 http://deb.debian.org/debian bookworm/main amd64 libglib2.0-0 amd64 2.74.6-2 [1398 kB]
Get:19 http://deb.debian.org/debian bookworm/main amd64 libglib2.0-data all 2.74.6-2 [1207 kB]
Get:20 http://deb.debian.org/debian bookworm/main amd64 libicu72 amd64 72.1-3 [9376 kB]
Get:21 https://download.docker.com/linux/debian bookworm/stable amd64 docker-buildx-plugin amd64 0.14.0-1~debian.12~bookworm [29.7 MB]
Get:22 http://deb.debian.org/debian bookworm/main amd64 libltdl7 amd64 2.4.7-5 [393 kB]
Get:23 https://download.docker.com/linux/debian bookworm/stable amd64 docker-ce-cli amd64 5:26.1.0-1~debian.12~bookworm [14.6 MB]
Get:24 http://deb.debian.org/debian bookworm/main amd64 libslirp0 amd64 4.7.0-1 [63.0 kB]
Get:25 http://deb.debian.org/debian bookworm/main amd64 libxml2 amd64 2.9.14+dfsg-1.3~deb12u1 [687 kB]
Get:26 http://deb.debian.org/debian bookworm/main amd64 patch amd64 2.7.6-7 [128 kB]
Get:27 http://deb.debian.org/debian bookworm/main amd64 shared-mime-info amd64 2.2-1 [729 kB]
Get:28 http://deb.debian.org/debian bookworm/main amd64 slirp4netns amd64 1.2.0-1 [37.5 kB]
Get:29 http://deb.debian.org/debian bookworm/main amd64 xdg-user-dirs amd64 0.18-1 [54.4 kB]
Get:30 https://download.docker.com/linux/debian bookworm/stable amd64 docker-ce amd64 5:26.1.0-1~debian.12~bookworm [25.2 MB]
Get:31 https://download.docker.com/linux/debian bookworm/stable amd64 docker-ce-rootless-extras amd64 5:26.1.0-1~debian.12~bookworm [9319 kB]
Get:32 https://download.docker.com/linux/debian bookworm/stable amd64 docker-compose-plugin amd64 2.26.1-1~debian.12~bookworm [12.4 MB]
Fetched 147 MB in 4s (36.1 MB/s)                 
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package pigz.
(Reading database ... 14162 files and directories currently installed.)
Preparing to unpack .../00-pigz_2.6-1_amd64.deb ...
Unpacking pigz (2.6-1) ...
Selecting previously unselected package less.
Preparing to unpack .../01-less_590-2_amd64.deb ...
Unpacking less (590-2) ...
Selecting previously unselected package libjansson4:amd64.
Preparing to unpack .../02-libjansson4_2.14-2_amd64.deb ...
Unpacking libjansson4:amd64 (2.14-2) ...
Selecting previously unselected package libnftnl11:amd64.
Preparing to unpack .../03-libnftnl11_1.2.4-2_amd64.deb ...
Unpacking libnftnl11:amd64 (1.2.4-2) ...
Selecting previously unselected package libnftables1:amd64.
Preparing to unpack .../04-libnftables1_1.0.6-2+deb12u2_amd64.deb ...
Unpacking libnftables1:amd64 (1.0.6-2+deb12u2) ...
Selecting previously unselected package nftables.
Preparing to unpack .../05-nftables_1.0.6-2+deb12u2_amd64.deb ...
Unpacking nftables (1.0.6-2+deb12u2) ...
Selecting previously unselected package xz-utils.
Preparing to unpack .../06-xz-utils_5.4.1-0.2_amd64.deb ...
Unpacking xz-utils (5.4.1-0.2) ...
Selecting previously unselected package apparmor.
Preparing to unpack .../07-apparmor_3.0.8-3_amd64.deb ...
Unpacking apparmor (3.0.8-3) ...
Selecting previously unselected package containerd.io.
Preparing to unpack .../08-containerd.io_1.6.31-1_amd64.deb ...
Unpacking containerd.io (1.6.31-1) ...
Selecting previously unselected package docker-buildx-plugin.
Preparing to unpack .../09-docker-buildx-plugin_0.14.0-1~debian.12~bookworm_amd64.deb ...
Unpacking docker-buildx-plugin (0.14.0-1~debian.12~bookworm) ...
Selecting previously unselected package docker-ce-cli.
Preparing to unpack .../10-docker-ce-cli_5%3a26.1.0-1~debian.12~bookworm_amd64.deb ...
Unpacking docker-ce-cli (5:26.1.0-1~debian.12~bookworm) ...
Selecting previously unselected package libip6tc2:amd64.
Preparing to unpack .../11-libip6tc2_1.8.9-2_amd64.deb ...
Unpacking libip6tc2:amd64 (1.8.9-2) ...
Selecting previously unselected package libnfnetlink0:amd64.
Preparing to unpack .../12-libnfnetlink0_1.0.2-2_amd64.deb ...
Unpacking libnfnetlink0:amd64 (1.0.2-2) ...
Selecting previously unselected package libnetfilter-conntrack3:amd64.
Preparing to unpack .../13-libnetfilter-conntrack3_1.0.9-3_amd64.deb ...
Unpacking libnetfilter-conntrack3:amd64 (1.0.9-3) ...
Selecting previously unselected package iptables.
Preparing to unpack .../14-iptables_1.8.9-2_amd64.deb ...
Unpacking iptables (1.8.9-2) ...
Selecting previously unselected package docker-ce.
Preparing to unpack .../15-docker-ce_5%3a26.1.0-1~debian.12~bookworm_amd64.deb ...
Unpacking docker-ce (5:26.1.0-1~debian.12~bookworm) ...
Selecting previously unselected package docker-ce-rootless-extras.
Preparing to unpack .../16-docker-ce-rootless-extras_5%3a26.1.0-1~debian.12~bookworm_amd64.deb ...
Unpacking docker-ce-rootless-extras (5:26.1.0-1~debian.12~bookworm) ...
Selecting previously unselected package docker-compose-plugin.
Preparing to unpack .../17-docker-compose-plugin_2.26.1-1~debian.12~bookworm_amd64.deb ...
Unpacking docker-compose-plugin (2.26.1-1~debian.12~bookworm) ...
Selecting previously unselected package libcurl3-gnutls:amd64.
Preparing to unpack .../18-libcurl3-gnutls_7.88.1-10+deb12u5_amd64.deb ...
Unpacking libcurl3-gnutls:amd64 (7.88.1-10+deb12u5) ...
Selecting previously unselected package liberror-perl.
Preparing to unpack .../19-liberror-perl_0.17029-2_all.deb ...
Unpacking liberror-perl (0.17029-2) ...
Selecting previously unselected package git-man.
Preparing to unpack .../20-git-man_1%3a2.39.2-1.1_all.deb ...
Unpacking git-man (1:2.39.2-1.1) ...
Selecting previously unselected package git.
Preparing to unpack .../21-git_1%3a2.39.2-1.1_amd64.deb ...
Unpacking git (1:2.39.2-1.1) ...
Selecting previously unselected package libglib2.0-0:amd64.
Preparing to unpack .../22-libglib2.0-0_2.74.6-2_amd64.deb ...
Unpacking libglib2.0-0:amd64 (2.74.6-2) ...
Selecting previously unselected package libglib2.0-data.
Preparing to unpack .../23-libglib2.0-data_2.74.6-2_all.deb ...
Unpacking libglib2.0-data (2.74.6-2) ...
Selecting previously unselected package libicu72:amd64.
Preparing to unpack .../24-libicu72_72.1-3_amd64.deb ...
Unpacking libicu72:amd64 (72.1-3) ...
Selecting previously unselected package libltdl7:amd64.
Preparing to unpack .../25-libltdl7_2.4.7-5_amd64.deb ...
Unpacking libltdl7:amd64 (2.4.7-5) ...
Selecting previously unselected package libslirp0:amd64.
Preparing to unpack .../26-libslirp0_4.7.0-1_amd64.deb ...
Unpacking libslirp0:amd64 (4.7.0-1) ...
Selecting previously unselected package libxml2:amd64.
Preparing to unpack .../27-libxml2_2.9.14+dfsg-1.3~deb12u1_amd64.deb ...
Unpacking libxml2:amd64 (2.9.14+dfsg-1.3~deb12u1) ...
Selecting previously unselected package patch.
Preparing to unpack .../28-patch_2.7.6-7_amd64.deb ...
Unpacking patch (2.7.6-7) ...
Selecting previously unselected package shared-mime-info.
Preparing to unpack .../29-shared-mime-info_2.2-1_amd64.deb ...
Unpacking shared-mime-info (2.2-1) ...
Selecting previously unselected package slirp4netns.
Preparing to unpack .../30-slirp4netns_1.2.0-1_amd64.deb ...
Unpacking slirp4netns (1.2.0-1) ...
Selecting previously unselected package xdg-user-dirs.
Preparing to unpack .../31-xdg-user-dirs_0.18-1_amd64.deb ...
Unpacking xdg-user-dirs (0.18-1) ...
Setting up libicu72:amd64 (72.1-3) ...
Setting up xdg-user-dirs (0.18-1) ...
Setting up libip6tc2:amd64 (1.8.9-2) ...
Setting up libglib2.0-0:amd64 (2.74.6-2) ...
No schema files found: doing nothing.
Setting up less (590-2) ...
Setting up libnftnl11:amd64 (1.2.4-2) ...
Setting up libcurl3-gnutls:amd64 (7.88.1-10+deb12u5) ...
Setting up libjansson4:amd64 (2.14-2) ...
Setting up liberror-perl (0.17029-2) ...
Setting up apparmor (3.0.8-3) ...
Created symlink /etc/systemd/system/sysinit.target.wants/apparmor.service → /lib/systemd/system/apparmor.service.
Setting up docker-buildx-plugin (0.14.0-1~debian.12~bookworm) ...
Setting up libglib2.0-data (2.74.6-2) ...
Setting up xz-utils (5.4.1-0.2) ...
update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode
Setting up containerd.io (1.6.31-1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service.
Setting up patch (2.7.6-7) ...
Setting up docker-compose-plugin (2.26.1-1~debian.12~bookworm) ...
Setting up libltdl7:amd64 (2.4.7-5) ...
Setting up docker-ce-cli (5:26.1.0-1~debian.12~bookworm) ...
Setting up libslirp0:amd64 (4.7.0-1) ...
Setting up pigz (2.6-1) ...
Setting up libnfnetlink0:amd64 (1.0.2-2) ...
Setting up git-man (1:2.39.2-1.1) ...
Setting up docker-ce-rootless-extras (5:26.1.0-1~debian.12~bookworm) ...
Setting up libxml2:amd64 (2.9.14+dfsg-1.3~deb12u1) ...
Setting up libnftables1:amd64 (1.0.6-2+deb12u2) ...
Setting up nftables (1.0.6-2+deb12u2) ...
Setting up slirp4netns (1.2.0-1) ...
Setting up shared-mime-info (2.2-1) ...
Setting up git (1:2.39.2-1.1) ...
Setting up libnetfilter-conntrack3:amd64 (1.0.9-3) ...
Setting up iptables (1.8.9-2) ...
update-alternatives: using /usr/sbin/iptables-legacy to provide /usr/sbin/iptables (iptables) in auto mode
update-alternatives: using /usr/sbin/ip6tables-legacy to provide /usr/sbin/ip6tables (ip6tables) in auto mode
update-alternatives: using /usr/sbin/iptables-nft to provide /usr/sbin/iptables (iptables) in auto mode
update-alternatives: using /usr/sbin/ip6tables-nft to provide /usr/sbin/ip6tables (ip6tables) in auto mode
update-alternatives: using /usr/sbin/arptables-nft to provide /usr/sbin/arptables (arptables) in auto mode
update-alternatives: using /usr/sbin/ebtables-nft to provide /usr/sbin/ebtables (ebtables) in auto mode
Setting up docker-ce (5:26.1.0-1~debian.12~bookworm) ...
invoke-rc.d: could not determine current runlevel
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket.
Processing triggers for libc-bin (2.36-9+deb12u6) ...

Starting jail docker with the following command:

systemd-run --property=KillMode=mixed --property=Type=notify --property=RestartForceExitStatus=133 --property=SuccessExitStatus=133 --property=Delegate=yes --property=TasksMax=infinity --collect --setenv=SYSTEMD_NSPAWN_LOCK=0 --unit=jlmkr-docker --working-directory=./jails/docker '--description=My nspawn jail docker [created with jailmaker]' --property=ExecStartPre=/mnt/xxxxxxx/jailmaker/jails/docker/.ExecStartPre -- systemd-nspawn --keep-unit --quiet --boot --bind-ro=/sys/module --inaccessible=/sys/module/apparmor --machine=docker --directory=rootfs --network-bridge=br1 --resolv-conf=bind-host '--system-call-filter=add_key keyctl bpf' --bind=/mnt/xxxxxxx/docker/:/mnt/docker --bind=/mnt/xxxxxxx/docker/data/:/mnt/data --bind=/mnt/xxxxxxx/docker/compose/:/mnt/compose --bind-ro=/mnt/xxxxxxx2/Storage:/mnt/Storage --bind-ro=/mnt/xxxxxxx2/Storage2:/mnt/Storage2

Running as unit: jlmkr-docker.service
root@xxxxxxx[~]# jlmkr shell docker
Connected to machine docker. Press ^] three times within 1s to exit session.
root@docker:~#

I just ran through the process myself, I never stumbled into the missing docker-compose dir. when you removed your old docker jail, did you destroy the old dataset as well?

I just ran through the process myself, I never stumbled into the missing docker-compose dir. when you removed your old docker jail, did you destroy the old dataset as well?

hm...

no.

i just delete the docker jail from shell.

jlmkr remove docker

but i didn't go into truenas dataset to delete there as well. i was tired and it didn't cross my mind xd

oh and ty so much Jip-Hop, Lockszmith-GH, dcwestra. you all made a difference. Also special thx to Stux who has been helping me the longest over at the truenas forum thread for this project. Oh anyone else that i missed.

newb question, new problem.

anyone know how to change the config to install a specific docker version? for truenas there was an issue with the latest docker apparently.

Apparently, docker 26.0.2 broke things. Downgrading to 26.0.1 fixes things.

https://forums.truenas.com/t/vm-network-does-not-work-since-installing-docker/2219/3

Well, you did say you were going to "lie down and rest", but then continued on ... 🤭

jlmkr remove... should take care of that, but maybe it didn't - that's my only thought.

Well, you did say you were going to "lie down and rest", but then continued on ... 🤭

jlmkr remove... should take care of that, but maybe it didn't - that's my only thought.

yeah but then crunch mode kicked in. Once i encounter a problem i can't stop though i really want to cauz it really bugged me. It's a curse ;(

OK, let's do this, close this issue, and open a discussion - that is the better convention for this type of help. An issue is for bugs in the script or for feature requests, discussion are a much more convenient channel.

OK, let's do this, close this issue, and open a discussion - that the better convention for this type of help. An issue is for bugs in the script or for feature requests, discussion are a much more convenient channel.

agreed. sorry that did cross my mind. plz proceed

Well, you did say you were going to "lie down and rest", but then continued on ... 🤭 jlmkr remove... should take care of that, but maybe it didn't - that's my only thought.

yeah but then crunch mode kicked in. Once i encounter a problem i can't stop though i really want to cauz it really bugged me. It's a curse ;(

I think that is true for all of us here. You are in the right place, my friend.

Jip-Hop / jailmaker

problem getting docker jailmaker to work #129