search

openshift / appliance

OpenShift-based Appliance Builder
Apache License 2.0
22 stars 27 forks source link

readme

OpenShift-based Appliance Builder

openshift-appliance is a command line utility for building a disk image that orchestrates OpenShift installation using the Agent-based installer. The primary use-case of the appliance is to support a fully disconnected installation of an OpenShift cluster. Thus, all required images are included in the appliance disk image.

Note: this README is intended for developers, usage details are available in the User Guide

Quick Start

Build (binary or container image)

Build binary

Install dependencies

Note: for oc-mirror usage, the builder ensures that the pull secret exists at ~/.docker/config.json

Build
make build-appliance

Build container image

export IMAGE=<image_url>

make build

Run

Commands

Flags

Create config file (appliance-config.yaml)

A configuration file named appliance-config.yaml is required for running the tool. The file should include the following properties:

Generate config file template

Using binary:

./build/openshift-appliance generate-config --dir assets

Using container image:

export IMAGE=<image_url>
export CMD=generate-config
export ASSETS=<assets-absolute-dir-path>

make run
Example
apiVersion: v1beta1
kind: ApplianceConfig
ocpRelease:
  version: 4.14
  channel: candidate
  cpuArchitecture: x86_64
diskSizeGB: 200
pullSecret: ...
sshKey: ...
userCorePass: ...

Start appliance disk image build flow

Using binary:

export LIBGUESTFS_BACKEND=direct

./build/openshift-appliance build --dir assets --log-level info

Using container image:

export IMAGE=<image_url>
export CMD=build
export ASSETS=<assets-absolute-dir-path>
export LOG_LEVEL=info/debug/error

make run
Cleanup

After a successful build, use the clean command before re-building the appliance (removes temp folder and state file).

Development

Running tests

skipper make test

Running lint

skipper make lint

Debug

Bootstrap step

Add --debug-bootstrap flag to the build command to avoid machine reboot on bootstrap step completion. Useful for taking a snapshot of the appliance disk image before testing changes in the install ignition.

unconfigured-ignition API

Add --debug-base-ignition flag to the build command for using a custom openshift-install binary to invoke agent create unconfigured-ignition. Use these instructions to build the openshift-install binary, and copy it into assets dir.

Test changes in the install ignition

To debug/test changes made in the InstallIgnition asset, follow the steps described on test_install_ignition.md

Main Components

Recovery ISO Assets (pkg/asset/recovery/)

Appliance Assets (pkg/asset/appliance/)

Ignition Assets (pkg/asset/ignition/)

High-Level Flow

The appliance build process consists of the following steps:

  1. Download CoreOS ISO
    • Extracted from the machine-os-images image (included in the release payload)
    • Used as the base image of the recovery ISO
  2. Generate recovery CoreOS ISO
    • Generated by embedding a custom bootstrap ignition to the base CoreOS ISO
    • Used as the recovery partition (labeled 'agentboot') of the appliance disk image
  3. Pull registry image
    • Used for serving the OCP release images on bootstrap and installation steps
  4. Pull release images required for bootstrap
    • Only a subset of the entire release payload (i.e. images that are required for bootstrap)
  5. Pull release images required for installation
    • Includes the entire release payload
  6. Generate data ISO
    • Includes the registry and release images that are pulled in previous steps
    • Used as the 'data partition' of the appliance disk image
  7. Download appliance base disk image
    • A qcow2 image of CoreOS
    • Used as the base disk image of the appliance
  8. Generate appliance disk image
    • Contains the following:
      • An ignition for orchestrating the OpenShift cluster installation
      • A recovery partition for reinstalling if necessary
      • The full OCP release payload for supporting disconnected environments

Demo

asciicast

Appliance disk image structure

$ virt-filesystems -a assets/appliance.raw -l -h
Name       Type        VFS      Label       Size  Parent
/dev/sda2  filesystem  vfat     EFI-SYSTEM  127M  -
/dev/sda3  filesystem  ext4     boot        350M  -
/dev/sda4  filesystem  xfs      root        180G  -
/dev/sda5  filesystem  iso9660  agentboot   1.2G  -
/dev/sda6  filesystem  iso9660  agentdata   18G   -

Configuration ISO

After booting a machine with appliance.raw disk image, the appliance looks for /media/config-image that should contain specific user values for cluster creation.

The config image is generated by invoking the following command:

$ openshift-install agent create config-image --dir assets

The assets directory should include install-config.yaml and agent-config.yaml files. For full details about the supported properties, see Agent-based Installer docs and the Installer user guide

The command outputs the following:

Note: for disconnected environments, specify a dummy pull-secret in install-config.yaml (e.g. '{"auths":{"":{"auth":"dXNlcjpwYXNz"}}}').

Examples

Creating an SNO cluster

agent-config.yaml
apiVersion: v1alpha1
kind: AgentConfig
rendezvousIP: 192.168.122.100
install-config.yaml
apiVersion: v1
metadata:
  name: appliance
baseDomain: appliance.com
controlPlane:
  name: master
  replicas: 1
compute:
- name: worker
  replicas: 0
networking:
  networkType: OVNKubernetes
  machineNetwork:
  - cidr: 192.168.122.0/24
platform:
  none: {}
pullSecret: '{"auths":{"":{"auth":"dXNlcjpwYXNz"}}}'
sshKey: 'ssh-rsa ...'

Creating a multi-node cluster

agent-config.yaml
apiVersion: v1alpha1
kind: AgentConfig
rendezvousIP: 192.168.122.100
install-config.yaml
apiVersion: v1
metadata:
  name: appliance
baseDomain: appliance.com
controlPlane:
  name: master
  replicas: 3
compute:
- name: worker
  replicas: 2
networking:
  networkType: OVNKubernetes
  machineNetwork:
  - cidr: 192.168.122.0/24
platform:
  baremetal:
    apiVIPs:
    - 192.168.122.200
    ingressVIPs:
    - 192.168.122.201
pullSecret: '{"auths":{"":{"auth":"dXNlcjpwYXNz"}}}'
sshKey: 'ssh-rsa ...'