kairos-io / kairos

:penguin: The immutable Linux meta-distribution for edge Kubernetes.
https://kairos.io
Apache License 2.0
1.13k stars 97 forks source link

Users aren't created when instantiated anywhere other than the top level. #1963

Open clanktron opened 1 year ago

clanktron commented 1 year ago

Below is the cloud config I used for testing:

#cloud-config
debug: true
hostname: nas
# fail on cloud config errors
strict: false
users:
  - name: test
    userid: 4004
    groups:
      - admin
      - docker
    passwd: mudlermudlermudler
    shell: /usr/bin/fish
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDOTdQLlqHFSdRU4iYNTx4Dgl+BUKnmSeV1od4BCvot0 clayton@ClaytonsMacBookPro.socal.rr.com
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG/1VzaIV0bnoIv//1FtbRSnwv5KE7KP/sgljykiqTLa mini
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOz0v4rsoL7f/A118ry+wWPd68pcvhkxrd0ITi8feUKQ mb-air
install:
  image: quay.io/clanktron/nas-ubuntu:5f1695b2397e61f18d6b1727275f868a06c84a9c
  ephemeral_mounts:
    - /scratch
  bind_mounts:
    - /var/lib/docker
    - /var/lib/tailscale
  extra-dirs-rootfs:
    - /data
    - /services
    - /scratch
    - /s3
  poweroff: true
  reboot: false
ssh_pwauth: false
timezone: America/Los_Angeles
stages:
  initramfs:
    - name: ensure users
      users:
        - name: clayton
          userid: 4000
          groups:
            - admin
            - docker
          lock_passwd: true
          shell: /usr/bin/fish
          ssh_authorized_keys:
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDOTdQLlqHFSdRU4iYNTx4Dgl+BUKnmSeV1od4BCvot0 clayton@ClaytonsMacBookPro.socal.rr.com
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG/1VzaIV0bnoIv//1FtbRSnwv5KE7KP/sgljykiqTLa mini
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOz0v4rsoL7f/A118ry+wWPd68pcvhkxrd0ITi8feUKQ mb-air
        - name: git
          userid: 4002
          lock_passwd: true
          shell: /usr/bin/git-shell
          ssh_authorized_keys:
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDOTdQLlqHFSdRU4iYNTx4Dgl+BUKnmSeV1od4BCvot0 clayton@ClaytonsMacBookPro.socal.rr.com
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG/1VzaIV0bnoIv//1FtbRSnwv5KE7KP/sgljykiqTLa mini
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOz0v4rsoL7f/A118ry+wWPd68pcvhkxrd0ITi8feUKQ mb-air
  boot:
    - name: really ensure users
      ensure_entities:
       -  path: /etc/passwd
          entity: |
                  kind: "user"
                  username: "foobar"
                  password: "barfoo"
                  uid: 4040
                  gid: 4040
                  homedir: "/home/foo"
                  shell: "/usr/bin/fish"

The image is based off of quay.io/kairos/core-ubuntu-22-lts:v2.4.1.

/etc/passwd after first install and reboot:

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
_apt:x:100:65534::/nonexistent:/usr/sbin/nologin
systemd-network:x:101:102:systemd Network Management,,,:/run/systemd:/usr/sbin/nologin
systemd-resolve:x:102:103:systemd Resolver,,,:/run/systemd:/usr/sbin/nologin
messagebus:x:103:104::/nonexistent:/usr/sbin/nologin
systemd-timesync:x:104:105:systemd Time Synchronization,,,:/run/systemd:/usr/sbin/nologin
sshd:x:105:65534::/run/sshd:/usr/sbin/nologin
Debian-snmp:x:106:111::/var/lib/snmp:/bin/false
_lldpd:x:107:112::/run/lldpd:/usr/sbin/nologin
_rpc:x:108:65534::/run/rpcbind:/usr/sbin/nologin
statd:x:109:65534::/var/lib/nfs:/usr/sbin/nologin
kairos:x:65535:65535:Created by entities:/home/kairos:/bin/bash
test:x:65536:65536:Created by entities:/home/test:/usr/bin/fish

Expected behavior: Users foobar, clayton, and git should all exist.

I also captured the output of the install with kairos-agent m config.yaml 2>&1 | tee kairos-agent.log so I can provide that if needed (its pretty long and I didn't see anything pertinent in the stderr/out).

Itxaka commented 1 year ago

@clanktron we would need the output of the intramfs stage which should be available after boot under /run/immucore/initramfs_stage.log (hint: if you add rd.immucore.debug to your cmdline it will store the debug output which is much nicer)

Also the output of journalctl -t cos-setup-boot to see the boot stage logs.

Thanks!

jimmykarily commented 1 year ago

The users: key inside the stage must be an object not an array:

This works:

#cloud-config

# https://github.com/kairos-io/kairos/issues/1963
debug: true
users:
  - name: kairos
    passwd: kairos

stages:
  initramfs:
    - name: ensure users
      users:
        clayton: 
          passwd: clayton
        git:
          passwd: git

This doesn't:

#cloud-config

# https://github.com/kairos-io/kairos/issues/1963
debug: true
users:
  - name: kairos
    passwd: kairos

stages:
  initramfs:
    - name: ensure users
      users:
        - name: clayton
          passwd: clayton
        - name: git
          passwd: git

The ensure_entities doesn't seem to apply but the syntax in the example above seems a bit different to the one here: https://github.com/mudler/yip#pushpin-yip . I'll dig deeper on this one.

jimmykarily commented 1 year ago

The problem with ensure_entities is the extra space before path in this:

       -  path: /etc/passwd

It should be:

  boot:
    - name: really ensure users
      ensure_entities:
       - path: /etc/passwd
         entity: |
                 kind: "user"
                 username: "foobar"
                 password: "barfoo"
                 uid: 4040
                 gid: 4040
                 homedir: "/home/foo"
                 shell: "/bin/sh"

With this config I see:

foobar:barfoo:65537:65537:Created by entities:/home/foo:/bin/sh

in /etc/passwd (I guess the password has to be encrypted and put in the shadow file)

Last thing, userid: in the top level users: section doesn't seem to have any effect. Not sure why we expect it to.

jimmykarily commented 1 year ago

Strange, the extra space before the path: doesn't seem to break any yaml validator I found, neither kairos-agent validate complains about it. Still it seems to create a problem, maybe the golang parser has a problem with it?

jimmykarily commented 12 months ago

Currently in kairos-agent and yip we use all 3 versions of yaml: gopkg.in/yaml.v[1|2|3]. I'll keep this open, to check if using v3 consistently everywhere would fix the extra space bug.

Itxaka commented 6 months ago

Agent was updated to only use yaml3. So only yip is missing the change to verify this

east4ming commented 4 months ago

Below is the cloud config I used for testing:

#cloud-config
debug: true
hostname: nas
# fail on cloud config errors
strict: false
users:
  - name: test
    userid: 4004
    groups:
      - admin
      - docker
    passwd: mudlermudlermudler
    shell: /usr/bin/fish
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDOTdQLlqHFSdRU4iYNTx4Dgl+BUKnmSeV1od4BCvot0 clayton@ClaytonsMacBookPro.socal.rr.com
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG/1VzaIV0bnoIv//1FtbRSnwv5KE7KP/sgljykiqTLa mini
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOz0v4rsoL7f/A118ry+wWPd68pcvhkxrd0ITi8feUKQ mb-air
install:
  image: quay.io/clanktron/nas-ubuntu:5f1695b2397e61f18d6b1727275f868a06c84a9c
  ephemeral_mounts:
    - /scratch
  bind_mounts:
    - /var/lib/docker
    - /var/lib/tailscale
  extra-dirs-rootfs:
    - /data
    - /services
    - /scratch
    - /s3
  poweroff: true
  reboot: false
ssh_pwauth: false
timezone: America/Los_Angeles
stages:
  initramfs:
    - name: ensure users
      users:
        - name: clayton
          userid: 4000
          groups:
            - admin
            - docker
          lock_passwd: true
          shell: /usr/bin/fish
          ssh_authorized_keys:
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDOTdQLlqHFSdRU4iYNTx4Dgl+BUKnmSeV1od4BCvot0 clayton@ClaytonsMacBookPro.socal.rr.com
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG/1VzaIV0bnoIv//1FtbRSnwv5KE7KP/sgljykiqTLa mini
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOz0v4rsoL7f/A118ry+wWPd68pcvhkxrd0ITi8feUKQ mb-air
        - name: git
          userid: 4002
          lock_passwd: true
          shell: /usr/bin/git-shell
          ssh_authorized_keys:
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDOTdQLlqHFSdRU4iYNTx4Dgl+BUKnmSeV1od4BCvot0 clayton@ClaytonsMacBookPro.socal.rr.com
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG/1VzaIV0bnoIv//1FtbRSnwv5KE7KP/sgljykiqTLa mini
            - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOz0v4rsoL7f/A118ry+wWPd68pcvhkxrd0ITi8feUKQ mb-air
  boot:
    - name: really ensure users
      ensure_entities:
       -  path: /etc/passwd
          entity: |
                  kind: "user"
                  username: "foobar"
                  password: "barfoo"
                  uid: 4040
                  gid: 4040
                  homedir: "/home/foo"
                  shell: "/usr/bin/fish"

The image is based off of quay.io/kairos/core-ubuntu-22-lts:v2.4.1.

/etc/passwd after first install and reboot:

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
_apt:x:100:65534::/nonexistent:/usr/sbin/nologin
systemd-network:x:101:102:systemd Network Management,,,:/run/systemd:/usr/sbin/nologin
systemd-resolve:x:102:103:systemd Resolver,,,:/run/systemd:/usr/sbin/nologin
messagebus:x:103:104::/nonexistent:/usr/sbin/nologin
systemd-timesync:x:104:105:systemd Time Synchronization,,,:/run/systemd:/usr/sbin/nologin
sshd:x:105:65534::/run/sshd:/usr/sbin/nologin
Debian-snmp:x:106:111::/var/lib/snmp:/bin/false
_lldpd:x:107:112::/run/lldpd:/usr/sbin/nologin
_rpc:x:108:65534::/run/rpcbind:/usr/sbin/nologin
statd:x:109:65534::/var/lib/nfs:/usr/sbin/nologin
kairos:x:65535:65535:Created by entities:/home/kairos:/bin/bash
test:x:65536:65536:Created by entities:/home/test:/usr/bin/fish

Expected behavior: Users foobar, clayton, and git should all exist.

I also captured the output of the install with kairos-agent m config.yaml 2>&1 | tee kairos-agent.log so I can provide that if needed (its pretty long and I didn't see anything pertinent in the stderr/out).

Sorry for another topic, I would like to know how to use kairos in conjunction with tailscale, can you help me? Thank you for your help.