TrueNAS Core 13 compatibility

[bernieke]

BurmillaOS Version: (ros os version) v1.9.6

Where are you running BurmillaOS? (docker-machine, AWS, GCE, baremetal, etc.) TrueNAS Core 13.0-U3 (bhyve)

Which processor architecture you are using? amd64

Do you use some extra hardware? (GPU, etc)? no

Some background:

FreeNAS 11 (TrueNAS's predecessor) came with a "docker VM host" capability, using RancherOS inside a bhyve VM.

These virtual machines can't be upgraded (they boot using grub from the FreeBSD side), and in TrueNAS the capability has meanwhile been removed. Which means these VMs are permanently stuck at RancherOS v1.4.2.

So I'd like to recreate these using the standard TrueNAS VM capability and BurmillaOS.

But I'm failing to get a virtual machine to even boot from the iso. UEFI reports BdsDxe: failed to load Boot0001 "UEFI BHYVE SATA DVD ROM BHYVE-BF16-952A-68C5" from PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0xFFFF,0x0): Not Found.

I've tried various options to create the VM, but only the "UEFI" boot method supports VNC, and I'm not getting anything on the serial console using either the "UEFI-CSM" (legacy BIOS) or "Grub" methods.

[olljanat]

I'm not familiar with FreeNAS but very quick look to https://www.truenas.com/community/resources/how-to-rancheros-on-freenas-11-0.50/ reminds me at least the fact that BurmillaOS installed still does not support UEFI #8

But it is possible to install it to UEFI if you follow those guides made by others.

[bernieke]

Ah, I see.

I'll just hang on to my v1.4.2 VMs for now, and revisit this when #8 has been resolved.

I guess I'll keep this ticket open until then, and close it if the UEFI support fixes it?

Thanks!

[olljanat]

That UEFI issue is not expected to be solved on anytime soon.

However if you have existing working RancherOS installation then you should be able to upgrade it by following (take backups first) https://burmillaos.org/docs/installation/upgrading/#upgrade-rancheros-to-burmillaos

I guess I'll keep this ticket open until then, and close it if the UEFI support fixes it?

Depends if there is other things which are needed because of FreeNAS. If not then you can just subscribe that UEFI issue.

[bernieke]

That doesn't work unfortunately, after the reboot it goes straight back to 1.4.2. It supposedly has something to do with the external grub on the FreeBSD side. But it might just be booting from the iso as well, I'm not sure how the FreeNAS guys implemented this back then.

I've subscribed to the UEFI ticket.

No big hurry on my part. Currently it's all working. And if it breaks, I can always deploy an Ubuntu VM and install docker there.

I just really like the minimalist and stateless approach of rancheros/burmillaos.

No idea if anything else would be needed besides UEFI... Ubuntu installs fine, so probably not...

[olljanat]

Ok. How you originally did that RancherOS installation?

[bernieke]

I didn't. It was a "docker host VM" feature FreeNAS had for a while. You just had to click a button to get a rancheros VM (pre-installed).

[olljanat]

Ok. Any idea about which version of FreeNAS it was then? (or which year so we can guess version)? FreeNAS old versions looks to be still available on https://download.freenas.org/ and http://archive.freenas.org/

because if we want to make sure that upgrade from old version to latest works we need use same grub config location here.

[bernieke]

FreeNAS 11.2 seems to have been the latest version with the feature. It was removed in 11.3

[olljanat]

Found actually their PR which originally added that feature https://github.com/truenas/middleware/pull/374 and their OS images looks to be stored to http://download.freenas.org/bhyve-templates/ Feature was removed by PR https://github.com/truenas/middleware/pull/2763

What I didn't found yet is that how those images are building but most probably we cane guess it by checking their content.

EDIT: Adding some debug commands here.

It is possible to boot from BurmillaOS ISO file and have their disk image available with command:

qemu-system-x86_64 \
  -curses \
  -net nic -net user,hostfwd=tcp::2222-:22 \
  -boot order=dc \
  -drive format=raw,media=cdrom,readonly,file=burmillaos-v1.9.7-rc1.iso \
  -drive format=raw,file=rancheros-bhyve-v1.4.2.img \
  -smp cores=4 -m 4G

After that it is possible to take SSH connection to that VM with command ssh -p 2222 rancher@localhost

And it is possible to boot directly from those images with command:

qemu-system-x86_64 \
  -curses \
  -net nic -net user,hostfwd=tcp::2222-:22 \
  -drive format=raw,file=rancheros-bhyve-v1.4.2.img \
  -smp cores=4 -m 4G

and then add rancher.password=... parameter on boot menu.