BMD

BMD (Bhyve Management Daemon) controls bhyve processes in order to VM configuration files. BMD reads all VM configuration files on the start up and boots VMs according to them.

When bmd received SIGTERM, bmd stops all running VMs by sending SIGTERM to bhyve processes and wait for them and then quit the daemon.

Sending SIGHUP to bmd triggers that bmd reloads all the VM configuration files. If boot parameter is changed, bmd boots or shutdown VM according to new boot parameter. If new VM config files are found, bmd boots VMs according to them. If bmd detects that some VM config files are deleted, bmd shutdown the VMs.

Bmd doesn't manage network environment to get avoid double management from FreeBSD rc scripts. All bridges must be created by the rc script. Bmd creates and destroys tap interfaces automatically and also assigns to the bridges. Uses need to specify which bridge interfaces are used for each VM.

If you prefer to use vale(4) switches for outer network connectivity, they must also be created and attached to a physical nic while the system starts. /etc/rc.local may help you. Bmd doesn't create tap interfaces in case of vale(4) use.

Bmd doesn't manage disk images, neither. Most of administrators have their own disk management policy and tools. So, it is better that disk images are created to follow the policy.

Requirements

1. grub2-bhyve package
2. edk2-bhyve package

Installation

$ make
$ sudo make install

If you want to install different path from /usr/local, set LOCALBASE environment variable like following.

$ export LOCALBASE=/opt/local
$ make
$ sudo make install
$ sudo make installconfig

Basic Usage

1. Enable the daemon

   service bmd enable

2. Starting the daemon

   service bmd start

3. Stopping the daemon

   service bmd stop

4. Reload VM config files

   service bmd reload

Logging

BMD writes log messages to LOG_DAEMON facility. Usually it is written in the /var/log/daemon.log file by the syslogd(8). You can see error messages of BMD and informational messages such as starting a VM or stopping a VM with or without an error.

BMD options

BMD can take following options. Write the options after "bmd_flags=" in /etc/rc.conf.

Configuration file

Bmd configuration file consists of 3 types of sections. Each section has key-value parameters and variables settings.

Sections

1. Global section

   Global section contains bmd options and global variables. The global section must be written in a file which root privileged user owns. If the other user owns the file, the global section is parsed but ignored.

2. Template section

   Template is a part of configuration for Virtual Machines. Configurations common to multiple VMs can be written in one template.

3. VM section

   Virtual Machine Configuration is written in this section.

Macros

Following 2 macros are available.

1. .apply

   This macro takes one or more template names to apply onto VM configuration. This macro can be written in template or vm sections.

2. .include

   This macro takes one file path to include another configurations. A file path can contain '*' and '?' literals for pattern matting. '.include' macro cannot be written in sections, must be written outside of section.

Variables

All variables belongs to global or local scope. A global scope variable is defined in global section and referred in all sections. A local scope variable is defined in VM section or template section and independent for each individual VM configurations. Variables defined in template section is available after applied from VM section. Before applying template, no variables are available written in template section. In template section, variables that defined before apply macro in VM section is available.

Bmd always sets following variables.

Arithmetic Calculation

Arithmetic calculations are performed by enclosing with $(( )). It's similar to /bin/sh. Numbers must be integers. Variables must contain integer number strings. Number format is decimal or octal or hexmal as same as C language. Supported operators are '+' '-' '*' '/' '%'. '(' and ')' are also available.

VM Configurations

Configurations are simply written "key = value;". If a key takes one or more values, multiple values can be written with comma separated. Or, '+=' operator can be used.

Following keys are available.

Global Configurations

Following keys are available for bmd.

Reloading configurations

Sending a HUP signal to the bmd triggers reloading VM configuration file(s). Each configuration will be applied when the VM restarts. Until restarting the VM, the old configuration will be kept. Please note that restarting the VM has 3 ways. One is the VM reboots spontaneously. This is a case reboot command is executed in the VM. In this case, poststop plugins (described below) won't be called so the old configuration will be kept after rebooting. Another ways are the VM is stopped by the bmdctl stop or reboot_on_change condition is satisfied. In these cases, bmd stops the VM and call poststop plugins and then the new configuration will be applied while restarting the VM.

Example configurations

Global Variables

global {
    $imgpath = /dev/zvol/zpool/images;
    $isopath = /zpool/iso;
}

Templates

template default_disk {
    disk = ${imgpath}/${NAME};
}

template graphics {
    graphics=yes;
    graphics_port=$((5900 + ${ID}));
    xhci_mouse=yes;
}

template serial {
    comport = auto;
}

template internet {
    network = bridge0;
}

template grub_inspect {
    loader=grub;
    loadcmd="auto";
    installcmd="auto";
}

FreeBSD Guest

vm freebsd {
    boot=yes;
    loader_timeout=15;
    ncpu=2;
    memory=2G;
    iso=${isopath}/FreeBSD-13.0-RELEASE-amd64-disc1.iso;
    loader=bhyveload;
    .apply default_disk, serial, internet;
}

NetBSD Guest

vm netbsd {
    boot=yes;
    ncpu=2;
    memory=2G;
    iso=${isopath}/NetBSD-9.2-amd64.iso;
    .apply default_disk, serial, internet, grub_inspect;
}

OpenBSD Guest

vm openbsd {
    boot=yes;
    ncpu=2;
    memory=2G;
    iso=${isopath}/OpenBSD-6.9-amd64.iso;
    .apply default_disk, serial, internet, grub_inspect;
}

CentOS Guest

vm centos {
    boot=yes;
    ncpu=2;
    memory=4G;
    #iso=${isopath}/CentOS-8.2.2004-x86_64-dvd1.iso;
    loader = uefi;
    .apply default_disk, internet, graphics;
}

Ubuntu Guest

vm ubuntu {
    boot=yes;
    ncpu=2;
    memory=4G;
    #iso=${isopath}/ubuntu-20.04.2.0-desktop-amd64.iso;
    loader=uefi;
    graphics_res=1280x720;
    .apply default_disk, internet, graphics;
}

Auto Inspection

When loadcmd or installcmd is set to auto, bmd inspects disk and iso images and generates loadcmd and installcmd values. This feature supports NetBSD and OpenBSD disk and iso images for now, and requires loader=grub;.

plugins

hook command plugin

Bmd calls the prestart plugin interface before starting a VM. Hook command plugin implements a prestart function that invokes a prestart command and waits for its termination. If the prestart command succeeds, the VM will boot otherwise won't. After VM termination, bmd calls the poststop plugin interface. Hook command plugin implements a poststop function that invokes a poststop command and waits for its termination. And then the VM state will change to STOP. Bmd also calls the on_status_change plugin interface when VM state is changed. Hook command plugin implements a on_status_change function that invokes a hookcmd. The hookcmd exit status will never affects anything.

The command line is as follows.

${prestart|poststop|hookcmd} ${vm name} ${state}

${state} is one of followings.

• LOAD

  loader is invoked

• RUN

  bhyve starts to run

• TERMINATE

  bhyve terminated

The sequence dialog is shown below. If grub-bhyve is used for the VM, replace bhyveload with grub-bhyve. If uefi is used, invoking bhyveload and hookcmd with LOAD state are skipped.

sequenceDiagram
  bmd->>+prestart: invoke
  prestart-->>-bmd: done
  loop if reboot
    bmd->>+bhyveload: invoke
    bmd->>+hookcmd: invoke with LOAD state
    hookcmd-->>-bmd: done
    bhyveload-->>-bmd: done
    bmd->>+bhyve: invoke
    bmd->>+hookcmd: invoke with RUN state
    hookcmd-->>-bmd: done
    bhyve-->>-bmd: done
  end
  bmd->>+poststop: invoke
  poststop-->-bmd: done
  bmd->>+hookcmd: invoke with TERMINATE state
  hookcmd-->>-bmd: done

avahi plugin

If VM is set graphics=yes and avahi-publish command is installed, Bmd publish remote frame buffer service under the VM name and vnc port. The publishing is kept while VM is running.

bmdctl

BMD control command via Unix Domain socket. Following subcommands are available.

Known Issues

1. Configuration keys are condensed compared to bhyve

It's under development.