Rebuild Raspbery PI - July 2023 Edition

[scottmuc]

Yay for Repaving!

As much as possible is documented inline in this issue template. In case of problems you may find help by viewing all the previous repave issues. Have fun!

Things to do with the existing build

• [x] Disable DHCP on the PI

  Instructions

  Ensure that when we renew our DCHP lease, it comes from our router.

  sudo systemctl stop dnsmasq

• [x] Enable DHCP on the router and remove port mapping and release/renew IP address

  Instructions

  Windows: ipconfig /release and then ipconfig /renew

• [x] Shutdown PI

  Instructions

  Make sure the USB drive has spun down before doing any work.

  sudo shutdown -h now

• [x] Create SD card with the latest Raspberry Pi OS

  Instructions

  Using the SD card in the now powered down PI.

  The new installer has options to enable SSH and create a user.

  installer download

  note check if the underlying Debian distribution is changing as this might result in some issues in the playbook execution.

  The Bullseye 64-bit lite image seems to work for now.

Post OS install steps on desktop

• [x] Ensure a working ansible enviroment

  Instructions

  This will exercise the asdf setup.

• [x] Turn on the PI and note the IP obtained from the Router

• [x] Clean up old host keys

  Instructions

  The new instance will have new host keys so to ensure host key warning messages don't distract us from the repaving, run the following:

  ssh-keygen -R 192.168.2.10
  ssh-keygen -R pi
  ssh-keygen -R pi.home.scottmuc.com
  ssh-keygen -R <pi ip>

• [x] Transfer local public ssh key to PI

  Instructions

  In order to avoid the use of sshpass, copy the current sessions public ssh key to to ./ssh/authorized_keys of the pi user on the PI. This user is only necessary to run the bootstrap playbook (which creates an admin ansible user) and will be subsequently cleaned up.

  ssh-copy-id pi@<pi ip>

• [x] Bootstrap with Ansible

  Instructions

  ./ansible.sh and select the bootstrap-playbook.yml

• [x] Add the PI port forwarding

  Instructions

  Needed for the certbot ACME challenge in the next step.

• [x] Complete full configuration

  Instructions

  ./ansible.sh and select the main-playbook.yml

• [x] Reboot PI

• [x] Re-add port mapping to the static IP

• [x] Disable DHCP on the router

• [x] Deploy goodenoughmoney.com

• [x] Create pi Samba user

  Instructions

  Run the following on the PI sudo smbpasswd -a smbrw

• [x] Make this template slightly better

How Do I Know I Am Done?

• [x] https://www.goodenoughmoney.com/ displays stuff

• [x] https://home.scottmuc.com/music/ loads navidrome and the music is playable

• [x] https://home.scottmuc.com/prometheus/ loads and has data

• [x] https://home.scottmuc.com/grafana/ loads and has data

• [x] Z:\ on my Windows PC works

• [x] ipconfig /release and then ipconfig /renew works

• [x] nslookup analytics.google.com is refused

• [x] Print out newly repaved machine details

  Instructions

  cat /etc/os-release && uname -a

[scottmuc]

Thoughts on DHCP Shuffling

Rather than the disable DHCPD on the PI and enable DHCPD on the router, I could statically assign an IP and DNS. It saves those 2 steps and I could live-stream a repave without the internet disruption on my PC.

That being said, when bootstrapping the PI, I need it to get an IP address so I can SSH to it. Perhaps there's a way to set the IP of the PI statically.

[scottmuc]

logkeepalive.sh Issue

Note that on reboot, /tmp/logkeepalive.sh gets deleted so the logging tag needs to be reapplied.

[scottmuc]

Repave issues

• grafana isn't working
• Alacritty clipboard isn't working

[scottmuc]

ansible@raspberrypi:~ $ cat /etc/os-release && uname -a
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 2023 aarch64 GNU/Linux

[scottmuc]

Resolving Issues

Grafana

The behaviour I was seeing was that going to https://home.scottmuc.com/grafana redirected to http://home.scottmuc.com:3000/grafana.

It looks like this repave brought in version 10 of grafana and that brought in a breaking change. I've temporarily locked the version to 9.5.6 and now it is working. A tiny bit of searching shows what looks to be the breaking change: https://github.com/grafana/grafana/pull/66724

I'll need to familiarize myself with that PR to see what I need to do. Version 10 was running but the reverse proxy configuration wasn't working.

Alacritty

Not sure what happened, but had to apply a configuration change for selections to be copied to the clipboard to work.

[scottmuc]

Repave complete! I'll edit/upload the video as soon as I can.

[scottmuc]

Here's the video (80min) of the whole exercise.