Expose a virtual BMC service per virtual machine, for IPMI and/or Redfish actions

[JM1]

Is your feature request related to a problem? Please describe: OpenShift and OpenStack numerous installers allow to deploy clusters to bare-metal servers and clouds. For development, debugging, testing etc. it helps to run these installers in local virtual environments. For example, a developer can deploy OpenShift and OpenStack on/to libvirt domains or qemu virtual machines running on her/his system locally. This requires setting up an BMC emulator such as virtualbmc (for IPMI) or sushy-emulator (for Redfish) which handles tasks such as virtual machine reboots and virtual media mounts.

Now, the overall goal is to Kubernetes-ize (containerize with K8s) these local virtualization-based deployments: Kubernetes+KubeVirt instead of libvirtd+virtualbmc/sushy-emulator. For example, setting up a k3s instance in Docker with KubeVirt and launching KubeVirt VMs is straight forward. What is missing though is a BMC emulator. Both virtualbmc and sushy-emulator require access to libvirtd, but iiuc KubeVirt's own virt-launcher is using virtqemud inside the VMI/virt-launcher pod only, so virtualbmc and sushy-emulator cannot be used.

When KubeVirt would provide a BMC emulator for each virtual machine, then we could deploy production-like (*) OpenShift or OpenStack clusters in KubeVirt VMs using their native bare-metal installers. Paired with k3s and Docker/Podman this would allow us to spin up and destroy a virtualized OpenShift or OpenStack cluster in a single container easily.

(*) Multi node, same cluster features, same installer and using IPMI/Redfish.

Describe the solution you'd like: A BMC emulator, speaking Redfish and/or IPMI protocols, should be provided by KubeVirt. KubeVirt should exposes this BMC emulator as a Kubernetes services to perform actions such as rebooting a virtual machine, changing its boot devices and mounting virtual media.

Describe alternatives you've considered:

• libvirt(d) based virtual machines with BMC emulators virtualbmc (for IPMI) or sushy-emulator (for Redfish).
• kcli allows to deploy various Kubernetes flavors in libvirtd based virtual machines.
  • Basically a wrapper for libvirt based approach.
• CRC allows to deploy a minimal OpenShift locally with Podman.
  • Subset of OpenShift features only.
  • No IPMI/Redfish installer.
• kind allows to deploy a Kubernetes cluster locally in Docker containers.
  • No OpenShift nor OpenStack support.
  • No IPMI/Redfish installer.
• OpenShift supports installation to public clouds such as Amazon, Azure, Google or OpenStack clouds.
  • Not a local deployment on developer machine.

[kubevirt-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[JM1]

/remove-lifecycle stale

[alancaldelas]

How would this work? I know nothing.

[kubevirt-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[JM1]

/remove-lifecycle stale

[kubevirt-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[JM1]

/remove-lifecycle stale

[kubevirt-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[fabiand]

I think it's a great idea to have a BMC for KubeVirt.

@colonwq iirc you worked on something like this on https://github.com/colonwq/virtualbmc ?

By today we'd probably want to have a redfish implementation.

@jm1 do you have capcity to contribute such a feature?

[JM1]

/remove-lifecycle stale

The OpenStack Ironic folks wrote a Redfish Emulator as part of sushy-tools. It offers a Redfish API and internally translates it to libvirtd. (OpenStack Ironic is also used in OpenShift, e.g. for IPI).

Some time ago I had a brief look of how this could be implemented in kubevirt (not so easy without access to libvirtd), but had to stop due to lack of time.

[JM1]

virtualbmc is similar to sushy-tools in the sense that it translates IPMI API to libvirtd calls.

[alancaldelas]

https://github.com/starbops/kubebmc

^ this seems like what we are trying to do right?

[JM1]

@alancaldelas Thank you for pointing us to KubeBMC 🤩 Indeed it answers this issue partially!

It uses VMware's Go IPMI library and its simulator in order to provide a subset of the IPMI API, mainly powering on/off a VM, querying the power state and setting the boot device to disk or pxe. For easier usage and distribution it would be great if it would be incorporated into KubeVirt.

For virtual media support, a Redfish API would still be necessary though.

[JM1]

A related tool is FakeFish:

a Flask based app that exposes a RedFish-like API with a set of limited endpoints [...]. The way it works is by running a set of scripts that interact with the hardware [...] while exposing a fake RedFish API [...]"

Recently it gained such scripts to interact with KubeVirt.

[fabiand]

I'd at least offer a KubeVirt T-Shirt if somebody is helping this project to come up with a RedFish implementation/backend for KubeVirt.

[kubevirt-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[JM1]

/remove-lifecycle stale

[xpivarc]

A proposal was merged, have the work on this started?

[kubevirt-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[JM1]

/remove-lifecycle stale