Initial Instructions to be automated

[CMCDragonkai]

What do we do to install NixOS again?

First we need to get wireless setup.

But we also need to install ZFS.

And parititon things.

First step we need network connectivity.

We don't have network manager.

So we need to use `wpa_supplicant.

# show the devices
ip link
# show info about the device
iw wlp58s0 info
# scan wireless access points
iw wlp58s0 scan
# connect to wireless (this will run it in background)
wpa_supplicant -B -i wlp58s0 -c <(wpa_passphrase 'TheFishburn' '*********')

Next we need to format the disks and setup the filesystem.

Before we can do that, we need to have ZFS available.

Install

nix-env -i git
nix-env -i mkpasswd

Get boot.supportedFilesystems = [ "zfs" ];. Now check the version:

cat /sys/module/zfs/version

Wait we also need the thing with 18.03. To do this. We need to update the channels.

Note that unetbootin only supports BIOS. Surely UEFI booting is possible?

To update the Nixos on the USB:

nix-channel --add https://nixos.org/channels/nixos-18.03 nixos
nix-channel --update
nixos-rebuild switch
# the above could fail if you don't have enough memory
# because it's not using disk, it's installing stuff into memory

This allows us to bring in the latest stuff first.

Alternatively just make sure to flash the latest version first.

Now we got nix, let's go ahead with it.

# show all the disks available (shows the persistent addresses)
ls /dev/disk/by-id

# wipe out the device
sgdisk --zap-all /dev/sda
# create EFI partition
sgdisk -n 1:0:+1GB -t 1:ef00 -c 1:"EFI System Partition" /dev/sda
# create disk partition
sgdisk -n 2:0:0 -t 2:8300 -c 2:"Main Data" /dev/sda

We are creating 1 GiB for the boot partition because we tend to end up with lots of builds of that style.

Next we can format our disk as ZFS.

cat /sys/class/block/sda/queue/physical_block_size # 512 or 4096
# since 512
# this means we use ashift=9

# we need to mount the /mnt

zpool create -f \
  -o ashift=9 \
  -o cachefile='' \
  -o altroot=/mnt \
  -o autoexpand=on \
  -o autoreplace=on \
  rpool /dev/disk/by-id/ata-WDC_WDS250G1B0B-00AS40_171812420165-part2

zfs set mountpoint=legacy rpool
zfs set compression=lz4 rpool
zfs set recordsize=128K rpool
zfs set primarycache=all rpool
zfs set secondarycache=all rpool
zfs set acltype=posixacl rpool
zfs set xattr=sa rpool
zfs set atime=on rpool
zfs set relatime=on rpool

mount -t zfs rpool /mnt

# create our rpool/tmp

zfs create \
    -o setuid=off \
    -o devices=off \
    -o sync=disabled \
    -o acltype=posixacl \
    -o xattr=sa \
    -o atime=on \
    -o relatime=on \
    -o primarycache=all \
    -o secondarycache=all \
    -o compression=lz4 \
    -o redundant_metadata=most \
    -o mountpoint=legacy \
    rpool/tmp

mkdir /mnt/tmp
mount -t zfs rpool/tmp /mnt/tmp
chmod 1777 /mnt/tmp

# copy over the cachefile (using the default location)

cp --parents /etc/zfs/zpool.cache /mnt

# format our boot disk

mkfs.fat -F 32 /dev/sda1
mkdir /mnt/boot
mount -t vfat /dev/sda1 /mnt/boot

Now we can start the installation!

We need to store everything.

nixos-generate-config --root /mnt

We didn't bother creating a swap. So 8 GiB is it. If you use more than this, it will result in errors. But we can address that later.

After this I was constructing the configuration.nix manually.

# set the password for root
passwd
# start sshd
systemctl start sshd

Setting up config.

rm -rf /mnt/etc/nixos
git clone https://github.com/MatrixAI/nixos-platforms.git /mnt/etc/nixos

mkdir /mnt/etc/nixos/intel-nuc7i7bnh/secrets
head -c4 /dev/urandom | od -A none -t x4 | tr -d ' \n' > /mnt/etc/nixos/intel-nuc7i7bnh/secrets/hostid
mkpasswd -m sha-512 | tr -d ' \n' > cmcdragonkai_password_hash
mkpasswd -m sha-512 | tr -d ' \n' > oliver_password_hash
mkpasswd -m sha-512 | tr -d ' \n' > vivian_password_hash

How to test if mkpasswd is correct? I don't know.

NIXOS_CONFIG="/mnt/etc/nixos/intel-nuc7i7bnh/configuration.nix" nixos-install

In the future, instead pin it in the configuration.nix. Instead of using the /nix/nixpkgs. Because it makes more sense. We pin the OS package set. Note that for changes to the OS, you can install packages without affecting the OS. But if you are making changes to the OS. A better way is overlays, or changing some aspect of pkgs. There's probably a way to override this. Best to find out.

After setting the root passwd. You can also then use:

vlock --all

Which will lock the screen and allow you tbuild and leave.

[CMCDragonkai]

At the end run:

bootctl --path=/mnt/boot install

Instead of making Nixos Configuration do it. It should be done manually for now in case you have different kinds of boot configurations.

Still haven't figured out how to best automate multiple disk configurations.

[CMCDragonkai]

Make sure to create /mnt/nix/nixpkgs and git clone nixpkgs there. And make sure to checkout the right one.

git clone https://github.com/NixOS/nixpkgs.git /mnt/nix/nixpkgs
cd /mnt/nix/nixpkgs
git remote add channels https://github.com/nixos/nixpkgs-channels
git fetch --all
git checkout -B channels-nixos-18.03 channels/nixos-18.03 

Also wifi driver is missing.

[CMCDragonkai]

To find out what WiFi driver you are using (at the USB boot) use:

readlink /sys/class/net/wlan0/device/driver

This tells me we are using iwlwifi. At version: 34.0.1.

[CMCDragonkai]

If you use startWhenNeeded ssh service sshd does not exist. Instead there's a sshd.socket which should mean that SystemD is listening on it. And will start and SSH service when needed. Use systemctl status sshd.socket.

[CMCDragonkai]

To reimport the pool while in boot usb, use zpool import -f rpool.

[CMCDragonkai]

The plugdev group is apparently still used by some udev rules. But they should be replaced by the systemd rules of using TAG+="uaccess", TAG+="udev-acl" tags in the udev rules. I'm adding it to all the extraGroups.

[CMCDragonkai]

Get this in asap! https://github.com/NixOS/nixpkgs/pull/39950 Then you can generate master GPG keys.

[CMCDragonkai]

Note that the plugdev group is historically used for USB devices. This has been superseded by using systemd and the uaccess and the udev-acl. I need to test if this works with ledger nano s.

[CMCDragonkai]

Creating swap on laptops should equal in total your RAM, as it would allow you to at least to do suspend to disk.

sgdisk -n 2:0:+16GB -t 2:8200 -c 2:"Swap" /dev/disk/by-id/...

The reason why we do a fixed partition swap is because ZFS Zvol swap is still not stable as of 0.7.9. Maybe not even 0.8.0, but at least native encryption will be available then.

[CMCDragonkai]

Rather than using this repo to clone, we should turn this into a config generator instead that can be loaded into a NixOS system and generate the config. Eventually it should also be embedded into a NixOS installation system (USB or Disk), and just be automated. We'd want Matrix AI Emergence to deal with this eventually. The main thing is that I don't want to clone this entire repo for a given system, yet each system will have its own repository of configuration.

[CMCDragonkai]

To prevent root password asking at the end:

nixos-install --no-root-passwd

I also found --no-channel-copy, perhaps this means there's no need to remove channels after rebooting. Not sure.

[CMCDragonkai]

The bootctl will automatically Linux Boot Manager record. But this isn't necessarily what we need when we have more complicated boot setups such as multiple boots.

efibootmgr \
  --create \
  --gpt \
  --disk /dev/disk/by-id/... \
  --part 1 \
  --label "Matrix EFI 1" \
  --loader /EFI/BOOT/BOOTX64.EFI

And create them appropriately. Also if you have 2 boot devices, you'll need to swap around and do the installation again in there.

Then we need to set the boot order:

efibootmgr --bootorder 0000,0001

Make sure the first one launched is always the first boot disk to be used inside the configuration.nix.

[CMCDragonkai]

When copying between 2 EFI system partitions we need:

dd if=/dev/disk/by-id/...-part1 of=/dev/disk/by-id/...-part1 status=none bs=128M

This ends up copying the disk UUID, so it's better to preserve the existing UUID.

UUID="1366-F51C"
printf "\x${UUID:7:2}\x${UUID:5:2}\x${UUID:2:2}\x${UUID:0:2}" \
| dd bs=1 seek=67 count=4 conv=notrunc of=/dev/disk/by-id/...-part1