Convert initial script to sdm [HDIK]

JapCLoN commented 12 months ago

Hello, here goes my initial script I have used for a long time, to get my pi's up and running with the default things I've needed.

#!/bin/bash

# Make sure we are root.
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 
   exit 1
fi

# Change APT proxy
echo "Adding APT proxy ..."
cat <<EOT >> /etc/apt/apt.conf.d/00aptproxy
Acquire::http::Proxy "http://10.0.0.120:3142";
EOT

# Disable swap, dhcp
echo "System is updated, disabling swap & dhcp ..."
service dphys-swapfile stop
systemctl disable dphys-swapfile
apt-get purge --remove dphys-swapfile bluez -y

# Make sure system is up to date
echo "Updating system, hang on ..."
apt update -y && apt upgrade -y && apt dist-upgrade -y && apt autoremove -y && apt clean

# Download needed packages
echo "I\'ll install the packages needed to run as wish"
apt install unattended-upgrades git python3-setuptools python-setuptools screen build-essential python-dev fail2ban ufw ntp php-cli php-curl apt-transport-https lsb-release ca-certificates curl rfkill -y

# Disable wifi and bt
echo "I\'ll disable wifi and bt"
rfkill block wifi
rfkill block bluetooth

# Setup static IP
echo "What\'s the last 3 digits of the IP?"
read ip

cat <<EOT >> /etc/dhcpcd.conf
interface eth0
static ip_address=10.0.0.$ip/24
static routers=10.0.0.1
static domain_name_servers=10.0.0.150
EOT

# Disable IPv6
cat <<EOT >> /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
EOT

echo "Device has gotten the static IP: 10.0.0.$ip"

# Set hostname
echo "Let\'s set a hostname for this device"
echo "What\'s the hostname?"
read hostname

hostn=$(cat /etc/hostname)
sed -i "s/$hostn/$hostname/g" /etc/hosts
sed -i "s/$hostn/$hostname/g" /etc/hostname

echo "Device has gotten the hostname: $hostname"

# Connect to NTP
echo "Connecting to NTP server"
echo "server ntp.server.local prefer iburst" > '/etc/ntp.conf'
echo "NTP has been added"

# add NAS mount
echo "DHCP has been disabled, will now attempt to mount NAS ..."
mkdir /mnt/01A
mkdir /mnt/03A
cat <<EOT >> /etc/fstab
//10.0.0.15/Share /mnt/01A cifs username=username,password=password,workgroup=WORKGROUP,uid=root,gid=root,vers=1.0,users,auto,user_xattr 0 0
nas.03a.server.local:/dc /mnt/03A nfs rw,vers=4  0 0
EOT
echo "NAS has been mounted"

# Let's tjek for update one last time.
echo "Checking for updates one last time before reboot ..."
apt update -y && apt upgrade -y && apt dist-upgrade -y && apt autoremove -y && apt clean

# Done, time to reboot
echo "We are all done now, rebooting system ..."
read -s -n 1 -p "Press any key to reboot"
reboot

JapCLoN commented 12 months ago

btw

# Change APT proxy
echo "Adding APT proxy ..."
cat <<EOT >> /etc/apt/apt.conf.d/00aptproxy
Acquire::http::Proxy "http://10.0.0.120:3142";
EOT

Is my apt-cacher-ng, can see you mention it in your video and just had to say that I do also recommend people to use it!

gitbls commented 12 months ago

Thanks for your interest in sdm!

Pre-apology: I did not actively test this, so if I duffed anything, just reply with the errors and we'll get them sorted.

That said, here are the steps to take to

Preparation
The sdm command to customize your IMG
The sdm command to burn your IMG to an SSD/SD Card
Code for your personal plugin
Code for a burn-time plugin to set a unique static address for each host

Preparation

Install sdm

curl -L https://raw.githubusercontent.com/gitbls/sdm/master/EZsdmInstaller | bash

Get a fresh, unmocified copy of the RasPiOS IMG you want to use. Save an unused copy so you can refresh your "working" copy if needed.

Notes:

sdm by default does an apt update, upgrade, and autoremove, so you won't see these in the notes below.
There is no built-in way in sdm to remove apt packages. That can be done in a personal plugin, which is at the bottom of this reply. This plugin does a couple of things:
- Removes those apt packages
- ntp: Updates your ntp configuration
- Creates the directories needed for your NAS mounts
Network configuration: See below.
user: You didn't say anything about setting up users. I've demonstrated below how you can delete the 'pi' user and add your own. Update myuser and mypassword as appropriate, or remove the plugin if you're not adding any users
Looks like you're using samba to mount a network share, but I don't see where you've installed samba. TBH I'm not sure offhand if it's needed to do the mount, but if it is, add it to the apps plugin below.
This assumes that you're running sdm on a Bookworm host. If not, the burn-time plugin will need to be changed to set the static IP differently.
Personally, I set my Pi static addresses in my DHCP server. If you did that, you would not need to set the IP address on each Pi, eliminating the need for the 2nd plugin myburnplugin.
There are many ways to implement functionality you desire in sdm. This note describes one way.

Pre-customization steps

These must be attended to before customizing your first IMG. These create files on your host system (where sdm runs) that are used during the customization process.

Create your eth0.nmconnection file

This note assumes your ethernet device is eth0. If it's something else, change eth0 as appropriate everywhere it's referenced.

You can either grab the .nmconnection file for your hardwired network from a running Bookworm system, or create one on a running Bookworm system.

The .nmconnection files are stored in /etc/NetworkManager/system-connections. Find the one for your hardwired ethernet and copy it to the host system where you're running sdm. Name it eth0.nmconnection (or whatever name you'd like to give it; I'll use eth0 here).

Make sure that in the .nmconnection file type=ethernet and device=eth0 (or whatever your device name is) are both set correctly.

To create one on a running Bookworm system:

nmcli --offline c add con-name eth0 type ethernet connection.interface-name eth0 > /path/to/eth0.nmconnection

Full NetworkManager documentation: https://networkmanager.dev/docs/api/latest/

Create your fstab addon

cat <<EOT >> /path/to/fstab.extension
//10.0.0.15/Share /mnt/01A cifs username=username,password=password,workgroup=WORKGROUP,uid=root,gid=root,vers=1.0,users,auto,user_xattr 0 0
nas.03a.server.local:/dc /mnt/03A nfs rw,vers=4  0 0
EOT

Customize the IMG

First you will customize the IMG. This establishes a common IMG for use across all your Pis. Settings that are Pi-specific, such as setting the IP address and hostname, are done when you burn the IMG to an SSD/SD Card.

sudo sdm --customize /path/to/IMG --aptcache 10.0.0.120 \
--plugin system:"service-disable=dphys-swapfile|fstab=/path/to/fstab.extension" \
--plugin apps:"apps=unattended-upgrades git python3-setuptools python-setuptools screen build-essential python-dev fail2ban ufw ntp php-cli php-curl apt-transport-https lsb-release ca-certificates curl rfkill" \
--plugin user:"deluser=pi|adduser=myuser|password=mypassword" \
--plugin disables:"bluetooth,wifi" \
--plugin network:"noipv6|nmconn=/path/to/eth0.nmconnection" \ 
--regen-ssh-host-keys \
--reboot

Burn the IMG to an SSD/SD Card

Once the customize successfully completes, burn the IMG to an SSD/SD card:

# Replace nnn with the desired IP address octet
sudo sdm --burn /dev/sdX --hostname yourhostname --plugin myburnplugin:"ipaddr=10.0.0.nnn" --expand-root /path/to/customized/IMG

Pop the disk into a Pi and boot it up. The system will NOT do the expand root (it will skip that relatively quick first boot). The system will go through the boot sequence, and at the end the sdm service FirstBoot (sdm-firstboot) will run. It finishes up any remaining tasks, such as regenerating the SSH host keys after the network connects, and then will reboot (--reboot).

When the system restarts from that, it is ready to go.

One cool, not completely obvious point: You'll use the same IMG when burning disks for each of your Pis. Their differences will be the things that you do on the burn command, such as setting the host name, and in this case, the IP address.

Personal plugin

There are two personal plugins here. The first (this one) is used during customization to do a couple of things that sdm doesn't have built-in constructs for.

The second one is used at burn-time to set the per-host static IP address.

#!/bin/bash
#
# This is an sdm plugin for: myplugin
#
# The plugin is called three times: for Phase 0, Phase 1, and post-install.
#

function loadparams() {
    source $SDMPT/etc/sdm/sdm-readparams
}

# $1 is the phase: "0", "1", or "post-install"
# $2 is the argument list: arg1=val1|arg2=val2|arg3=val3| ...
#
# Main code for the Plugin
#
phase=$1
pfx="$(basename $0)"     #For messages
args="$2"
loadparams
vldargs=""
rqdargs=""

if [ "$phase" == "0" ]
then
    #
    # In Phase 0 all references to directories in the image must be preceded by $SDMPT
    #
    logtoboth "* Plugin $pfx: Start Phase 0"
    plugin_getargs $pfx "$args" "$vldargs" "$rqdargs"
    logtoboth "* Plugin $pfx: Complete Phase 0"

elif [ "$phase" == "1" ]
then
    #
    # Phase 1 (in nspawn)
    #
    logtoboth "* Plugin $pfx: Start Phase 1"
    plugin_getargs $pfx "$args" "$vldargs" "$rqdargs"
    #
    #
    logtoboth "Plugin $pfx: Remove packages I don't want"
    apt-get purge --remove dphys-swapfile bluez --yes
    # Connect to NTP \
    logtoboth "Plugin $pfx: Connecting to NTP server"
    echo "server ntp.server.local prefer iburst" > '/etc/ntp.conf'
    logtoboth "Plugin $pfx: Create shares for fstab mounts"
    mkdir /mnt/01A
    mkdir /mnt/03A
    logtoboth "* Plugin $pfx: Complete Phase 1"
else
    #
    # Plugin Post-install edits
    #
    logtoboth "* Plugin $pfx: Start Phase post-install"
    plugin_getargs $pfx "$args" "$vldargs" "$rqdargs"
    logtoboth "* Plugin $pfx: Complete Phase post-install"
fi

Personal burn-time plugin

This plugin is used on the sdm --burn command to enable you to give each Pi a unique static IP address.

Place this file (using sudo) in /usr/local/sdm/local-plugins.

#!/bin/bash
#
# This is an sdm plugin for: myburnplugin
#
# The plugin is called three times: for Phase 0, Phase 1, and post-install.
#

function loadparams() {
    source $SDMPT/etc/sdm/sdm-readparams
}

# $1 is the phase: "0", "1", or "post-install"
# $2 is the argument list: arg1=val1|arg2=val2|arg3=val3| ...
#
# Main code for the Plugin
#
phase=$1
pfx="$(basename $0)"     #For messages
args="$2"
loadparams
vldargs="|ipaddr|"
rqdargs="|ipaddr|"

if [ "$phase" == "0" ]
then
    #
    # In Phase 0 all references to directories in the image must be preceded by $SDMPT
    #
    logtoboth "* Plugin $pfx: Start Phase 0"
    plugin_getargs $pfx "$args" "$vldargs" "$rqdargs"
    logtoboth "* Plugin $pfx: Complete Phase 0"

elif [ "$phase" == "1" ]
then
    #
    # Phase 1 (in nspawn)
    #
    logtoboth "* Plugin $pfx: Start Phase 1"
    plugin_getargs $pfx "$args" "$vldargs" "$rqdargs"
    logtoboth "> Plugin $pfx: Set static address $ipaddr"
    nmcli --offline c mod eth0 ipv4.addresses $ipaddr < /etc/NetworkManager/system-connections/eth0.nmconnection > /etc/NetworkManager/system-connections/eth0.nmconnection
    logtoboth "* Plugin $pfx: Complete Phase 1"
else
    #
    # Plugin Post-install edits
    #
    logtoboth "* Plugin $pfx: Start Phase post-install"
    plugin_getargs $pfx "$args" "$vldargs" "$rqdargs"
    logtoboth "* Plugin $pfx: Complete Phase post-install"
fi

gitbls commented 12 months ago

btw
# Change APT proxy
echo "Adding APT proxy ..."
cat <<EOT >> /etc/apt/apt.conf.d/00aptproxy
Acquire::http::Proxy "http://10.0.0.120:3142";
EOT
Is my apt-cacher-ng, can see you mention it in your video and just had to say that I do also recommend people to use it!

Indeed. sdm and apt-cacher-ng are a wonderful combination!

gitbls commented 11 months ago

Hi, just checking in to see if you've had an opportunity to have a go at this, and are you having any questions/problems?

gitbls commented 11 months ago

Closing this issue due to no activity. @JapCLoN, please feel free to re-open this issue if you need further assistance/guidance.

gitbls / sdm