[CERTTF-316] Use attachments for provisioning (muxpi)

Description

Build on Testflinger support for file attachments (CERTTF-303) to introduce provisioning using a file attachment instead of downloading an image from a URL. In the scope of this PR this only applies to the muxpi device connector.

The file attachment to be used is specified in the new use_attachment field, provided as an alternative to the url field. Previously, url was required, now one of url or use_attachment are required.

Significant refactoring/reimplementation has taken place in this context: previously, downloading the image from the url, decompressing and flashing on the device media was entirely performed on the device connector using a series of pipes, roughly like this:

# device connector
curl -sf {url} | zstdcat | sudo dd of={test_device}"

This was efficient and avoided saving the downloaded image file on the device connector (where storage is generally constrained). However, it also made logging and debugging difficult, as the whole process was consolidated, making it hard to distinguish which part failed.

In this PR, the process is initiated on the agent and is roughly performed like this:

# agent    | # device connector
cat {path} | ssh {ssh_options} {control_user}@{control_host} "zstdcat | sudo dd of={test_device}"

The path here is the provisioning image, which has either been previously downloaded on the agent from a URL or is an attachment.

This unifies the process of transferring the image on the device connector and flashing it, regardless of the source of the image, and also allows for more fine-grained logging and debugging.

Resolved issues

Linked to CERTTF-316.

Documentation

The section regarding attachments in the Testflinger reference has been updated.

Tests

All tests performed on the rpi4b1g001 agent on testflinger-staging, with the muxpi device connector installed from this branch.

Provisioning from a URL

job_queue: rpi4b1g-001
provision_data:
  url: http://cdimage.ubuntu.com/ubuntu-core/22/dangerous-stable/current/ubuntu-core-22-arm64+raspi.img.xz

See result.

Provisioning from an attachment

job_queue: rpi4b1g-001
provision_data:
  attachments:
    - local: /home/boukeas/Downloads/busybox-gadget_1.0_dx-series.kernos.zst
      agent: busybox-gadget_1.0_dx-series.kernos.zst
  use_attachment: busybox-gadget_1.0_dx-series.kernos.zst
  create_user: false
  boot_check_url: http://$DEVICE_IP:8888/v1/health

See result.

Error cases: Provisioning without a URL or from a non-existent attachment

job_queue: rpi4b1g-001
provision_data:
  attachments:
    - local: /home/boukeas/Downloads/busybox-gadget_1.0_dx-series.kernos.zst
      agent: busybox-gadget_1.0_dx-series.kernos.zst
  create_user: false
  boot_check_url: http://$DEVICE_IP:8888/v1/health

See result.

job_queue: rpi4b1g-001
provision_data:
  attachments:
    - local: /home/boukeas/Downloads/busybox-gadget_1.0_dx-series.kernos.zst
      agent: busybox-gadget_1.0_dx-series.kernos.zst
  use_attachment: busybox-gadget_1.0_dx-series.kernos
  create_user: false
  boot_check_url: http://$DEVICE_IP:8888/v1/health

See result.

canonical / testflinger