search

nanovms / ops

ops - build and run nanos unikernels
https://ops.city
MIT License
1.27k stars 132 forks source link

add virtualbox integration #888

Closed am11 closed 3 years ago

am11 commented 3 years ago

Hi, I am looking for some clarification on https://github.com/nanovms/ops#build-a-bootable-image. I started with the earlier hi.js example from the readme on Ubuntu running in a VM, then:

afterwards, created a VM using hi.vdi in VirtualBox and started the VM. It gives the following error (followed by reboot and this repeats indefinitely):

image

Is it supposed to boot in isolation with VirtualBox like that?

fabioDMFerreira commented 3 years ago

The image must be built by ops. Locally you can run the images on qemu or hyper-v if you use windows.

As you're trying to run a nodejs program I suggest you run the next command

$ ops load node_v14.2.0 -a hi.js

Check more nodejs examples here https://github.com/nanovms/ops-examples/tree/master/nodejs

am11 commented 3 years ago

With qemu, the hi.img built from Linux works on both Linux and on macOS (copied over from Linux due to ELF binary constraint of ops-build command):

# ✅ successfully ran webserver using hi.img on macOS and Linux
$ qemu-system-x86_64 -machine q35 \
    -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x3 \
    -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x3.0x1 \
    -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x3.0x2 \
    -device virtio-scsi-pci,bus=pci.2,addr=0x0,id=scsi0 \
    -device scsi-hd,bus=scsi0.0,drive=hd0 \
    -device isa-debug-exit \
    -device virtio-net,bus=pci.3,addr=0x0,netdev=n0,mac=76:e0:e4:cd:d4:0b \
    -no-reboot -cpu max -vga none -machine q35 \
    -m 2G -drive file=$HOME/.ops/images/hi.img,format=raw,if=none,id=hd0 \
    -netdev user,id=n0,hostfwd=tcp::8083-:8083 \
    -display none -serial stdio

However, I am interested in running in VirtualBox as my setup is primarily using this stack (or docker containers). Moreover, I was successfully able to run unik generated image in VirtualBox, but unik seems to be unmaintained for couple of years now.

am11 commented 3 years ago

A few findings.. if we convert the hi.img to VirtualBox format VDI -- using VBoxManage convertdd (see full command in first post) or qemu-img convert -f raw -O vdi ~/.ops/images/hi.img ~/.ops/images/hi.vdi -- it gives the error as shown in the screenshot above. Found that we have two options, either:

both options independently fix the crash-reboot-repeat issue.

The next error occurs right after the boot from HDD:

-- snip --
00:00:01.805034 VMMDev: Guest Log: BIOS: Boot : bseqnr=1, bootseq=0042
00:00:01.806121 VMMDev: Guest Log: BIOS: Booting from Hard Disk...
00:00:02.288075 GUI: UIMachineViewNormal::resendSizeHint: Restoring guest size-hint for screen 0 to 800x600
00:00:02.288108 ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={4680b2de-8690-11e9-b83d-5719e53cf1de} aComponent={DisplayWrap} aText={The console is not powered up}, preserve=false aResultDetail=0
00:00:02.295321 GUI: 2D video acceleration is disabled
00:00:02.295451 GUI: HID LEDs sync is enabled
00:00:02.330975 GUI: UIMachineLogicNormal::sltCheckForRequestedVisualStateType: Requested-state=0, Machine-state=5
00:00:03.330101 GIM: KVM: VCPU  0: Enabled system-time struct. at 0x0000000000070000 - u32TscScale=0xd5555558 i8TscShift=-1 uVersion=2 fFlags=0x1 uTsc=0xdd6b0517 uVirtNanoTS=0x5c41ecca TscKHz=2399998
00:00:03.330150 TM: Switching TSC mode from 'Dynamic' to 'RealTscOffset'
00:12:12.404107 Console: Machine state changed to 'Stopping'
00:12:12.411468 Console::powerDown(): A request to power off the VM has been issued (mMachineState=Stopping, InUninit=0)
00:12:12.411896 ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={4680b2de-8690-11e9-b83d-5719e53cf1de} aComponent={DisplayWrap} aText={The console is not powered up}, preserve=false aResultDetail=0
00:12:12.413752 Changing the VM state from 'RUNNING' to 'POWERING_OFF'
-- snip --

it shows up in log file and screen stays blank indefinitely (no reboots).

This is potentially a feature request for ops/lepton/vdi_image.go to translate raw (.img) to a usable VDI format. 😄 Or a working example in docs of a simple hello-world app image generated by ops-build, and successfully run in VirtualBox. Seems like there is a VirtualBox-related bullet in this tracking issue: https://github.com/nanovms/nanos/issues/336, which wasn't checked.

eyberg commented 3 years ago

we currently don't have any virtual box support - we could add it if really desired but qemu/hvf works pretty well already - have you tried that?

am11 commented 3 years ago

Yup, I have tried nanos with qemu hypervisor. However, on-premise virtual box images seamlessly integrate with oracle cloud infrastructure OCI, as well as dev workflows which primarily use virtual box provider with vagrant.

Statistically, in vagrant world, https://app.vagrantup.com/boxes/search shows that virtualbox is the most popular one; out of 22 supported providers. So I do not see it as the odd-one-out product, but the opposite. :)

It seems like we have enough abstractions in place. If OOTB virtual box support can be trivially achieved, please do so; that would save folks mixing different stacks in their dev environments.

francescolavra commented 3 years ago

VirtualBox support is now in Nanos master branch, and should be available in the next nightly release.

eyberg commented 3 years ago

can this be closed @am11 ?

am11 commented 3 years ago

@eyberg, I was thinking this is tracking --target virtual-box and #577 is tracking --target oci (similar to --target hyper-v and --target azure; two separate options in ops).

eyberg commented 3 years ago

k - so we have initial OCI support - I'm going to re-purpose this tkt for vbox integration since nanos supports it now - we'd just need to add some image-create/instance-create