Create a random mfsbsd password / support ssh public key

[grembo]

Right now, running the script listens to the public internet with fixed, known, weak credentials.

It would be cool if the script could generate a random password for mfsbsd and show it on start. This way the security risk for the server would be greatly reduced.

Another great feature would be supporting an ssh public key (and therefore using no password at all) and/or limiting access by IP address.

[bretton]

It would be cool if the script could generate a random password for mfsbsd and show it on start.

it's possible to set the password on building the mfsBSD image, however I'm not seeing a practical way to generate a random one every time the ISO boots.

This would have to be a value-add, with users generating, or requesting, a custom mfsBSD ISO image at depenguin.me

Another great feature would be supporting an ssh public key (and therefore using no password at all)

This is also possible as part of the build process for the mfsBSD image. It could be a generic (and compromised) keypair, or it could be user-supplied and a custom image generated.

This would have to be a value-add, with users generating, or requesting, a custom mfsBSD ISO image at depenguin.me

and/or limiting access by IP address.

This might be possible as part of the qemu-static step for hostfwd, and could be a parameter that's passed in.

[grembo]

I have an idea how to do ssh public key quite easily - simply make the mfsbsd image pull the key file either from the host qemu is running on (eg using http) or place it on a filesystem that is mounted in there. Then copy in place on image boot. Should be pretty straightforward (just adding a startup script to the mfsbsd image). Password should be set to * in master.passwd. The password could be set on using the same principle, but ssh public key is nicer.

[bretton]

We don't have any mount in or copy in on boot. We have password-based ssh, and maybe limit to localhost on boot?

perhaps there could be a step in this script, before the qemu-system-x86_64 command, which opens a background process, which keeps trying to ssh to localhost:1022 with known password, and on success:

• copy over authorized_keys from rescue system to /root/.ssh/authorized_keys in qemu image and/or fetches authorized_keys file from http source to /root/.ssh/authorized_keys in qemu image
• disables ssh password authentication
• restarts ssh process

it would need to happen in background because the qemu-system-x86_64 doesn't exit. Or maybe the qemu-system-x86_64 command can go to the background, and then the ssh connect loop starts?

[bretton]

I've updated to copy in ssh key from local file or url with 86cbdb799a00535dd1cad4b6d2d05756e4dc07c0

Tested in local setup and can remotely login without password. To be tested with Hetzner.

[grembo]

Hm, this still has the basic problem of being available publicly using a well-known password for root, which means using it to maintain a server that contains critical data is a security risk (it's enough for someone to fire at that IP every few seconds until they are lucky and plant something in there).

I had to play with ubuntu yesterday and found a howto for vm-bhyve, which has exactly the properties I would like to see for depenguin-me:

1. No passwords at all
2. Log in using a non-root user (then sudo)
3. SSH public key can be specified on provisioning

This is how it was created:

vm img http://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img
vm create -c 16 -m 32G -t linux-zvol -i focal-server-cloudimg-amd64.img -C -k ~/.ssh/id_rsa.pub ubuntu
vm start

What happens under the hood is that vm-bhyve is creating a cloud-init iso, which is picked up by the OS in the vm (similar to what I suggested above "simply make the mfsbsd image pull the key file either from the host qemu is running on (eg using http) or place it on a filesystem that is mounted in there.", but instead of being Michael's hack, cloud-init is a standard).

See https://github.com/churchers/vm-bhyve/blob/master/lib/vm-core#L254 for how this works at the vm-bhyve end.

---

You could do something very similar (with or without cloud-init)

1. Alter depenguin script to create automatically an ISO containing the ssh public key(s) => basically an authorized_keys file
2. Alter depenguin script to connect this ISO as a CDROM drive to qemu
3. Alter your mfsBSD image in the following ways (either altering the stock or creating a new one):
   • Remove root password (pw usermod -h -)
   • Disable root login in sshd_config (PermitRootLogin no)
   • Add a cloud-init-(like) start script that does on firstboot (which is every boot in this case):
     • Adduser mfsbsd without a password, invite to group wheel

       pw groupadd mfsbsd
       pw useradd -m -n mfsbsd -g mfsbsd -G wheel -h - -c "mfsbsd user"

     • Install and configure sudo:

       pkg install -y sudo
       cat > /usr/local/etc/sudoers.d/wheel<<EOF
       %wheel ALL=(ALL) NOPASSWD: ALL
       EOF

     • Mount the CD Drive:

       mount_cd9660 /dev/cdrom /mnt

     • Copy the ssh public key into ~/mfsbsd/.ssh/authorized_keys:

       mkdir -p ~mfsbsd/.ssh
       cp /mnt/mfsbsd_authorized_keys ~mfsbsd/.ssh/authorized_keys
       chmod 644 ~mfsbsd/.ssh/authorized_keys

     • Unmount the CD Drive

       umount /mnt

This way, there is never a moment when root access without a private SSH key is possible. If anything goes wrong in provisioning, no access the granted (the secure default). Instead of writing the script above, you could also take a look at net/cloud-init, but what I outlined above might already be just fine and better suited to the task at hand.

[bretton]

exactly the properties I would like to see for depenguin-me:

1. No passwords at all
2. Log in using a non-root user (then sudo)
3. SSH public key can be specified on provisioning

Thanks, will review further!

The mfsBSD img/iso can have ssh keys added as part of the build process which generates the ISO.

This is currently done separately to the depenguin script, and in advance. (see my fork at https://github.com/depenguin-me/depenguin-installer)

The depenguin script downloads a statically-compiled qemu binary, and mfsbsd iso, and loads the prebuilt image as the installer using qemu. And now injects an SSH key as well.

I don't know how we'd get a linux rescue system to build the mfsbsd image as part of the script (requires freebsd).

To make it work (mostly) as you describe, the depenguin script could:

• create a local keypair, upload it with compromised user/pass (restrict access to host IP somehow)
• connect to ssh with key on localhost, update ssh keys and settings, restart ssh, run other commands to make user access possible
• run installer, install package cloud-init, prepare cloud-init config
• disable root ssh access, change root pass
• reboot

then when it reboots it does so with user access via key, and maybe with cloud-init prepped for further steps.

Right now, the depenguin script gives a freebsd rescue system or installer. Installation still needs to happen manually, user can configure as needed. It can replace the missing rescue console at Hetzner.

Alternatively, I could explore a facility where people can add their pubkeys and click a button to have an image built for them, available for 24h, where they copy-paste the shown command, and they can install from a temporary mfsbsd image which includes their pubkeys? i.e. click to generate your custom installer, then use it within 24 hours.

[grembo]

Alternatively, I could explore a facility where people can add their pubkeys and click a button to have an image built for them, available for 24h, where they copy-paste the shown command, and they can install from a temporary mfsbsd image which includes their pubkeys? i.e. click to generate your custom installer, then use it within 24 hours.

But why not do what I described? This could be done in one static image that never has to change and allows the user to pass in an ssh key. Very straightforward - I could provide a proof on concept based on the depenguin-installer repo (and maybe try to upstream this to mfsbsd as a configuration option later, so you won't have to maintain a fork in the long run). Simple, elegant and really secure (no passwords involved, no known keys involved, no firewalling required).

[bretton]

But why not do what I described?

the statically compiled qemu binary loads in the mfsbsd image as cdrom mount, doesn't seem possible to add another, might be a way

if we load mfsbsd img as disk (not ISO!) it messes with the passed in disks. Would need to experiment somewhat.

[grembo]

According to https://wiki.gentoo.org/wiki/QEMU/Options, it should be possible to specify cdrom multiple times:

-drive - Advanced configuration of a virtual CDROM drive: -drive file=IMAGE.iso,media=cdrom - Set a virtual CDROM drive and use the specified image file for it. With this syntax you can set multiple drives.

[bretton]

In principle your suggestion works. -drive file=IMAGE.iso,media=cdrom will load as /dev/cd0 and the mfsbsd install will be on /dev/cd1 when called as -cdrom /path/to/mfsbsd-iso

busy with testing and then will update the live site. Should be good to do an install test in an hour or so

[grembo]

So would

-drive file=mfsbsd.iso,media=cdrom  -drive file=image.iso,media=cdrom 

make mfsbsd.iso /dev/cd0 and the image containing setup info (like the ssh authorized keys bunde) /dev/cd1?

(instead of -cdrom /path/to/mfsbsd-iso -drive file=IMAGE.iso,media=cdrom)

Or wouldn't qemu boot from the mfsbsd ISO in this case? (or doesn't it matter anyway, as only one of the ISOs is bootable?)

[bretton]

-drive file=mfsbsd.iso,media=cdrom  -drive file=image.iso,media=cdrom 

I had issues with this. Was easier to mount each cd device and check for file needed, then process.

Could be tested more thoroughly and refined, but for now the script is working and I'm keen to see confirmation an install worked.

Please try from within rescue console with link to pubkey:

wget https://depenguin.me/run.sh && chmod +x run.sh && ./run.sh -n "http://host.dom/keys.pub"

There's an awful delay on em0 up and before SSH available due to rc.local processing. If first ssh connect to installer has issues, wait a bit and try again.

[grembo]

Hi Bretton,

I tried it and it works in general. Boot time was actually good, first connection to ssh worked. This is starting to look really promissing! :+1:

Notes:

• rc.local has a syntax error, /usr/sbin/pw usermod -h - must be /usr/sbin/pw usermod root -h - (username missing), therefore root has a password. BUT I think that the password should be removed while provisioning the image originally, not in rc.local, as this leaves a small gap where login could be possible. If I'm reading the mfsbsd sources correctly, this could be as easy as setting ROOTPW_HASH="*" in the build.
• Likewise, /etc/sshd/sshd_config still says PermitRootLogin yes - This should be removed during image creation (there's probably a hook for it somewhere). Edit: The easiest way to fix this is probably creating a custom script in (see https://github.com/mmatuska/mfsbsd/blob/42b8c42093e6a61252edb0c37fb7b6c2ca95b60b/Makefile#L473). Might make sense to place everything that can be done at image build time there. Keeping things like pw usermod in rc.local can't hurt though, as a second line of defense.
• run.sh needs a little patch to work with "-k ":

--- run.sh.orig 2022-07-31 20:48:56.000000000 +0200
+++ run.sh  2022-08-01 11:24:16.783715666 +0200
@@ -23,7 +23,7 @@
     case "${flags}" in
         k)
             AUTHKEYURL=""
-            AUTHKEYFILE=${OPTARG}
+            AUTHKEYFILE=$(realpath "${OPTARG}")
             ;;
         n)
             AUTHKEYURL=${OPTARG}

[grembo]

Addendum: I tried on Hetzner with their version of QEMU which is a bit newer and it worked ok. (QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-11+deb11u1))

It requires a few changes to parameters (they did a few syntax changes), but that's fairly simple. I'll give you more details later.

[bretton]

* rc.local has a syntax error, `/usr/sbin/pw usermod -h -` must be `/usr/sbin/pw usermod root -h -`

fixed in https://github.com/depenguin-me/depenguin-installer/commit/f4d27c329310c53c07434844eae9bc7b05452d09 and rebuild happening

If I'm reading the mfsbsd sources correctly, this could be as easy as setting ROOTPW_HASH="*" in the build.

I set a password in my build script, or it defaults to mfsroot. I can add that.

* Likewise, /etc/sshd/sshd_config still says `PermitRootLogin yes` 

this is set in my build script

# custom configs
if [ ! -f conf/rc.conf ]; then
    cp -f conf/rc.conf.sample conf/rc.conf
    sysrc -qf conf/rc.conf ifconfig_DEFAULT="DHCP"
    sysrc -qf conf/rc.conf sshd_flags="-oUseDNS=no -oPermitRootLogin=no"
fi

I was doing several sed changes to sshd_config in rc.local but then had issues logging in, so removed them while diagnosing that. I can add back the general hardening steps as with packer images.

* run.sh needs a little patch to work with "-k ":

thanks! fixed https://github.com/depenguin-me/mfsbsd-13.1-script/commit/8169dae58775ad06ecd87f57453dbe2c27c8cf16

[grembo]

I set a password in my build script, or it defaults to mfsroot. I can add that.

Yeah, if you don't set ROOTPW and set ROOTPW_HASH="" that should do the trick (parameter to make, as in `make -DROOTPW_HASH=""` <- untested)

[grembo]

I was doing several sed changes to sshd_config in rc.local but then had issues logging in, so removed them while diagnosing that. I can add back the general hardening steps as with packer images.

Something like customfiles/harden.sh:

pw usermod root -h -
sed -i "" "s/^PermitRootLogin.*//g" SOMEPATH/sshd_config

(not sure what SOMEPATH would be relative to the image)

[bretton]

working with SHA256 (mfsbsd-13.1-RELEASE-amd64.iso) = 849432e7dc98bcacfc744a163e9df09ee8059ec3d369fb4f89be51f1c365cc1c on the site, using build parameter ROOTPW_HASH="*"

[bretton]

first bash at a consumer-builder: https://github.com/depenguin-me/depenguin-builder

[bretton]

going to close this