qemu: add internal/external storage

[jakogut]

Allow for booting works from external storage, with blank internal storage available for flashing. This will enable flasher images to be tested without unwrapping.

Change-type: patch Signed-off-by: Joseph Kogut joseph@balena.io

[klutchell]

@jakogut I don't see any obvious issues, but I also don't understand everything as well as you do.

Were you able to run a successful test on Jenkins? Are any changes required in the suites or Leviathan core to support this change?

[jakogut]

@klutchell Tests pass in Jenkins. There should be no modifications required downstream.

[klutchell]

This repo needs to be switched to Flowzone, which will take me a little while to sort out. I'll revisit this PR when it's unblocked.

[jakogut]

@balena-ci I self-certify!

[klutchell]

Depends on https://github.com/balena-os/leviathan-worker/pull/36

[jakogut]

Thanks for your help with this, Kyle!

[klutchell]

It seems to be failing the test on generic-aarch64 but that same test passed on a few other PRs today.

Is it because generic-aarch64 is not a flasher?

leviathan-worker-worker-1  | KVM is unavailable, falling back on software emulation
leviathan-worker-worker-1  | QEMU args:
leviathan-worker-worker-1  |  [
leviathan-worker-worker-1  |   '-nographic',
leviathan-worker-worker-1  |   '-m',
leviathan-worker-worker-1  |   '1G',
leviathan-worker-worker-1  |   '-smp',
leviathan-worker-worker-1  |   '1',
leviathan-worker-worker-1  |   '-serial',
leviathan-worker-worker-1  |   'file:/reports/dut-serial.txt',
leviathan-worker-worker-1  |   '-drive',
leviathan-worker-worker-1  |   'format=raw,file=/data/internalDisk.img,media=disk',
leviathan-worker-worker-1  |   '-drive',
leviathan-worker-worker-1  |   'format=raw,file=/data/externalDisk.img,if=none,id=ext0',
leviathan-worker-worker-1  |   '-device',
leviathan-worker-worker-1  |   'qemu-xhci',
leviathan-worker-worker-1  |   '-device',
leviathan-worker-worker-1  |   'usb-storage,drive=ext0,bootindex=1',
leviathan-worker-worker-1  |   '-M',
leviathan-worker-worker-1  |   'virt',
leviathan-worker-worker-1  |   '-cpu',
leviathan-worker-worker-1  |   'cortex-a72',
leviathan-worker-worker-1  |   '-net',
leviathan-worker-worker-1  |   'nic,model=e1000',
leviathan-worker-worker-1  |   '-net',
leviathan-worker-worker-1  |   'bridge,br=br5ccv5e87',
leviathan-worker-worker-1  |   '-bios',
leviathan-worker-worker-1  |   '/usr/share/qemu/edk2-aarch64-code.fd',
leviathan-worker-worker-1  |   '-qmp',
leviathan-worker-worker-1  |   'tcp:localhost:5000,server,nowait',
leviathan-worker-worker-1  |   '-vnc',
leviathan-worker-worker-1  |   ':0',
leviathan-worker-worker-1  |   '-device',
leviathan-worker-worker-1  |   'virtio-gpu-pci'
leviathan-worker-worker-1  | ]
leviathan-worker-client-1  | [2022-12-14T20:19:42.155Z][worker-e2e] DUT powered on
leviathan-worker-client-1  | [2022-12-14T20:19:42.157Z][worker-e2e] Creating SSH tunnels to DUT
leviathan-worker-client-1  | [2022-12-14T20:19:42.159Z][worker-e2e] Getting ip of dut
leviathan-worker-client-1  | [2022-12-14T20:49:43.244Z][worker-e2e]     # Subtest: Power Cycle Tests
leviathan-worker-client-1  | [2022-12-14T20:49:43.245Z][worker-e2e]         # Subtest: Is the DUT reachable?
leviathan-worker-client-1  |             not ok 1 - 500 - "Could not resolve f5152ee.local"

[jakogut]

@klutchell These changes should be compatible with flasher and raw images, on both x86_64 and aarch64, but I'll test on aarch64 real quick and make sure.

Basically, with non-flasher images, we're emulating booting the raw image from a USB thumb drive.

[jakogut]

@klutchell It looks like the problem is the generic-aarch64 kernel builds with CONFIG_USB_XHCI_PCI=m, so the root disk isn't available in the initramfs. We need to either compile this built-in, or add the module to the initramfs. I'm looking into this now.

[jakogut]

Requires: https://github.com/balena-os/balena-generic/pull/178

[jakogut]

@resin-jenkins retest this please

[klutchell]

The above device PR needs to be released to production before this will pass tests.

It's also not a Jenkins job, just GitHub actions so you can re-run jobs from the actions tab in the UI. @resin-jenkins has no power here!

On Fri, Dec 16, 2022, 4:38 p.m. Joseph Kogut @.***> wrote:

@resin-jenkins https://github.com/resin-jenkins retest this please

— Reply to this email directly, view it on GitHub https://github.com/balena-os/leviathan-worker/pull/35#issuecomment-1355658768, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE4CWIDW6F7T6JJSKIV5HDDWNTONDANCNFSM6AAAAAASZWGUYU . You are receiving this because you were mentioned.Message ID: @.***>

[klutchell]

It appears that 2.108.1 is the current production release for this device, which includes your change so I would expect this to pass.

However the last failed run shows an earlier release in the logs so I guess I will kick off a re-run.

leviathan-worker-client-1  | [2022-12-18T18:47:11.180Z][worker-e2e] Unpacking the file: /data/downloads/balenaOs-latest.img
leviathan-worker-client-1  | [2022-12-18T18:47:11.181Z][worker-e2e] Logging into balena with balenaSDK
leviathan-worker-client-1  | [2022-12-18T18:47:11.241Z][worker-e2e] Logged in with testbot's account on balena-cloud.com using balenaSDK
leviathan-worker-client-1  | [2022-12-18T18:47:11.441Z][worker-e2e] Checking version in os-release
leviathan-worker-client-1  | [2022-12-18T18:47:11.456Z][worker-e2e] Found version in os-release file: 2.107.13+rev1

[klutchell]

Not bumping the worker version in leviathan core as HUP tests wont pass until this change is old enough to be the "previous" production release as the HUP FROM version.