Failing to boot in Vmware Fusion 12

[mszoek]

This may be the cause: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251866

[grahamperrin]

VMWare Fusion

Yes, almost certainly FreeBSD bug 251866:

… Because the loader.efi modified the size of EFI_STAGING_SIZE, vmware could not start the system above FreeBSD 12.2

– although if https://cgit.freebsd.org/src/commit/?id=b304cd9789ca7ff3df629af42a976450e8660a11&h=stable%2F12 alone did not fix it, then maybe the bug should have a different title.

VirtualBox

Helium_0.2.1-F12.2_h0.5.0-amd64.iso does boot in VirtualBox 6.1.22 on FreeBSD 14.0-CURRENT:

[mszoek]

Thanks @grahamperrin, that's good to know! I also switched to using VirtualBox on macOS for development. It runs in Qemu with HVF acceleration as well, but neither of these are as fast/seamless as VmWare Fusion :(

I'm not sure if this failure is actually from that upstream bug. My initial versions of Helium were built using the FreeBSD Makefile targets to produce an ISO and memstick image, and just included an additional helium.txz in the base OS. Those (and vanilla FreeBSD 12.2) worked fine in Fusion. The boot failures started when I switched to building it with the helloSystem ISO generator which is based on FuryBSD.

Hopefully I can dig into this after I finish #46 !

[mszoek]

Hmm. I rebased to FreeBSD branch stable/12 which has the fix for EFI_STAGING_SIZE, but the problem still exists.

I think this is a different issue altogether, since the initial kernel on the ISO boots and the uzip file is streamed into a ZFS ramdisk. It is after this, during the startup of the live image, that Vmware aborts.

[mszoek]

Verbose boot with Legacy BIOS (happens with EFI too - just the last line says efirtc0 instead):

By using boot -v and setting boot_mute="NO" in loader.conf, I can then see errors about missing /compat after the Linux ELF exec handler is loaded, followed by a kernel panic:

[mszoek]

Another one:

[mszoek]

Nailed it! The problem is with the dsbdriverd package which is installed to enable automatic hardware detection. Looks like probing the buses on Fusion causes a panic :( Setting dsbdriverd_enable="NO" in loader.conf stops the panics but not sure yet what functionality is lost. Perhaps OK for the short term.

[grahamperrin]

https://github.com/mszoek/airyx/issues/47#issuecomment-846615356

… a different issue altogether, since the initial kernel on the ISO boots …

That certainly differs from FreeBSD 251866. I hid my earlier comment.

[grahamperrin]

https://github.com/mszoek/airyx/issues/47#issuecomment-847133859

… Setting dsbdriverd_enable="NO" in loader.conf stops the panics but not sure yet what functionality is lost. …

Re: https://github.com/mrclksr/DSBDriverd#readme I guess that with real hardware (not virtualised), there'll be less automated driving of things that are not (automatically) driven by a pre-configuration of the OS.

With which ISO do you get the panics?

I wonder whether a panic can be made reproducible with VirtualBox …

[mszoek]

With which ISO do you get the panics?

Any Airyx/Helium ISO had this issue once I switched to using the helloSystem ISO build script. It includes and enables the dsbdriverd package as part of the build. The panic happens when this service starts up inside the ramdisk of the Live CD and (presumably) probes the virtual bus or devices in some way, or loads a particular driver.

I suspect I could install dsbdriverd on a vanilla FreeBSD 12.2 in Fusion and trigger the same issue.

I wonder whether a panic can be made reproducible with VirtualBox …

Possibly. So far it is isolated to Vmware Fusion.

[probonopd]

Can the issue be reproduced with NomadBSD?

Possibly it should be reported over at https://github.com/mrclksr/DSBDriverd/issues

[mszoek]

I'll check whether Nomad has the issue.

So far I have adjusted furybsd-init-helper to avoid running dsbdriverd under Vmware (only), which gets around the problem and should not affect real hardware installs or other hypervisors.