boot: setup node - approaches to use iPXE to boot SmartOS

[AmitKumarDas]

Approaches to using iPXE to boot SmartOS:

Since my approach basically comes down to 2 components, an Nginx frontend, and a backend manager for pulling down and mounting the platform archive to somewhere nginx can reach it relative to the iPXE script, Kubernetes offered a pretty straightforward way to do this.

I only needed to create a base Docker image that would contain my iPXE script, and an informational webpage:

#!ipxe
dhcp
set base-url http://sdc-ipxe.east.gourmet.yoga
kernel ${base-url}/smartos/smartos/platform/i86pc/kernel/amd64/unix -B smartos=true,console=ttyb,ttyb-mode="115200,8,n,1,-"
module ${base-url}/smartos/smartos/platform/i86pc/amd64/boot_archive type=rootfs name=ramdisk
boot

and the Dockerfile to build this image:

FROM nginx
COPY index.html /usr/share/nginx/html/index.html
COPY smartos.ipxe /usr/share/nginx/html/smartos.ipxe

Build and push it to the registry.

The tricky part, and where Kubernetes could be the most help, would be where I pull down the archive, place it on a volume, and then mount it to my above Nginx container. I did this by having an InitContainer pull the archive and decompress it to a volume (I used a hostPath, but since this is not data I wish to persist between schedulings, an emptyDir would suffice as well):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ipxe-deployment
  labels:
    app: ipxe
spec:
  replicas: 2
  selector:
    matchLabels:
      app: ipxe
  template:
    metadata:
      labels:
        app: ipxe
    spec:
      containers:
...
      initContainers:
      - name: config-data
        image: ubuntu:xenial
        command: ["/bin/sh","-c"]
        args: ["apt update; apt install -y wget tar; wget https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/platform-latest.tgz; tar xvf platform-latest.tgz -C /data; mkdir /data/smartos; mv /data/platform* /data/smartos/platform"]
        volumeMounts:
        - mountPath: /data
          name: ipxe-data
      volumes:
      - name: ipxe-data
        hostPath:
          path: /mnt/kube-data/ipxe/
          type: DirectoryOrCreate
      imagePullSecrets:
      - name: regcred

The above has, before my Nginx container is deployed, declared that an InitContainer will run first to extract the archive to the volume, and once that has completed, will proceed to create the Nginx container: spec:

      containers:
      - name: ipxe
        image: coolregistryusa.biz/jmarhee/smartos-ipxe:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 80
        volumeMounts:
        - mountPath: /usr/share/nginx/html/smartos
          name: ipxe-data

So, once the InitContainer completes, the archive volume will be mounted to /usr/share/nginx/html/smartos which is where the ipxe script expects the path to the kernel to be. The last piece is the Kubernetes Service to expose this to the network, so, if you use a provider like Packet (see the above linked pieces for consuming this endpoint) to boot using Custom iPXE, you can boot from this endpoint:

kind: Service
apiVersion: v1
metadata:
  name: ipxe-service
spec:
  selector:
    app: ipxe
  ports:
  - protocol: TCP
    port: 8085
    targetPort: 80
  type: LoadBalancer

ref - https://medium.com/@jmarhee/using-kubernetes-to-provide-ipxe-infrastructure-for-up-to-date-smartos-platform-897bf4f2cb4