Try to remove kernel args as part of hyperkit process

crc-org / crc

CRC is a tool to help you run containers. It manages a local OpenShift 4.x cluster, Microshift or a Podman VM optimized for testing and development purposes

https://crc.dev

Apache License 2.0

1.26k stars 241 forks source link

Try to remove kernel args as part of hyperkit process #2182

Open praveenkumar opened 3 years ago

praveenkumar commented 3 years ago

As of now we provide the ostree info as part of kernel argument for hyperkit which is part of crc metadata and updated during the bundle creation. Since with 4.8 openshift going to support single node which means user able to run the machine config changes and after reboot instance should be on latest ostree layer. In case of hyperV and libvirt this is not an issue but hyperkit we have to find something so we don't relay on this kernel args.

cfergeau commented 3 years ago

Is the issue the following?

4.8 crc (maybe) runs MCO
user does configuration changes through MCO
the ostree=/ostree/boot.0/rhcos/bf3b38268ddb2a2070dc587b361ce45c46a6f9e3606bbd3b5e15b2e6d3d47e5d/0 boot parameter in the kernel command line changes

Since the kernel parameters are currently hardcoded in the bundle metadata on macOS, this cannot work.

praveenkumar commented 3 years ago

@cfergeau That's right, from 4.8 side user can enable MCO and do config changes which MCO take care and update the ostree.

cfergeau commented 3 years ago

With hyperkit, we could use UEFI boot to remove the need for kernel/initrd/kernel cmdline, but I'm not sure the same is going to be possible with virtualization.framework.

guillaumerose commented 3 years ago

Using qemu on macOS would solve that.

It might be a good move. I heard multiple times that it might be better than the Virtualization framework.

Also: https://docs.docker.com/docker-for-mac/apple-m1/#docker-desktop-rc-3

In this build, we have defaulted to a qemu-based virtual machine, which we believe resolves some of the issues noted as known issues in the previous release candidate. You can switch between qemu-based and virtualization.framework-based virtual machines using the Preferences

praveenkumar commented 3 years ago

That's good to know, we should at least evaluate it and check what issues we might face.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

cfergeau commented 3 years ago

Might become a more pressing issue with the MCO changes coming in 4.8

cfergeau commented 3 years ago

Using UEFI is not complicated:

download a UEFI.fd build (docker desktop has one, and https://freebsd.pkgs.org/12/freebsd-amd64/uefi-edk2-bhyve-0.2_1,1.txz.html worked too)
apply this small hyperkit driver patch: https://github.com/cfergeau/crc-driver-hyperkit/commit/4e6a85f7d19725aa05bbef6468cc4dc3c531d321

cfergeau commented 2 years ago

I came across https://github.com/cloud-hypervisor/rust-hypervisor-firmware the other day, I wonder if this (or a similar project) could be useful to solve this issue:

The purpose is to be able to use this firmware to be able to load a bootloader from within a disk image without requiring the use of a complex firmware such as TianoCore/edk2 and without requiring the VMM to reuse functionality used for booting the Linux kernel.

This can be used with qemu's --kernel:

qemu-system-x86_64 [...] -kernel ./target/target/release/hypervisor-fw [...]