Spike rootfs build no ssh

[radekg]

Spike implementation for https://github.com/combust-labs/firebuild/issues/38.

[radekg]

Next steps on this PR:

• [x] test dependency resources for multi-stage builds
• [x] integrate the bootstrap server into the rootfs command
• [x] bootstrap metadata for vminit
• [x] vminit changes to handle bootstrap metadata: https://github.com/combust-labs/firebuild-mmds/pull/3

[radekg]

At this stage bootstrapping Consul 1.9.4 via MMDS is working. The VM connects to the secure gRPC bootstrap server and bootstraps successfully. Resulting rootfs can be launched from, app starts as expected.

Next steps:

• [x] extract all TODO marked settings to rootfs flags
• [x] work out proper logging for the VM output sent via gRPC
• [x] trace relevant remote bootstrap server and embedded CA operations
• [x] documentation

[radekg]

The vminit v0.0.10 is the first version where the MMDS bootstrap works fine for Alpine and Debian. It bootstraps Postgres 13, etcd 3.4.0, Consul 1.9.4 and Traefik 2.4.7.

There is a problem fetching direct HTTP resources which was a special case to handle and needs to be handled properly. The direct HTTP resource is downloaded to the host build temporary directory and should be transferred to the bootstrapper via gRPC. But it seems this isn't happening so some test coverage for such case needs to be added. The only trace in logs is:

2021-04-03T23:34:59.986Z [DEBUG] vminit.executing-deployer: executing ADD command: command="{<nil> ADD https://dl.minio.io/server/minio/release/linux-amd64/minio /usr/bin/minio /mnt/sde1/fc/cache/builds/bplnfgdsc2wd8f2qnfhk/sources/minio/latest/Dockerfile https://dl.minio.io/server/minio/release/linux-amd64/minio /usr/bin/minio { /} { 0:0} <nil>}"

Welcome to Alpine Linux 3.13
Kernel 5.8.0 on an x86_64 (ttyS0)

192 login: 2021-04-03T23:35:00.650Z [DEBUG] rootfs.build-server.grpc-impl: sending data with safe buffer size: traceId=6023c7afeea4e88d resource=/usr/bin/minio safe-buffer-size=3774873
2021-04-03T23:35:04.143Z [DEBUG] vminit.bootstrap: pinging server
2021-04-03T23:35:04.241Z [DEBUG] rootfs: received ping from bootstrap client: traceId=6023c7afeea4e88d
2021-04-03T23:35:09.147Z [DEBUG] vminit.bootstrap: pinging server
2021-04-03T23:35:09.245Z [DEBUG] rootfs: received ping from bootstrap client: traceId=6023c7afeea4e88d
2021-04-03T23:35:14.151Z [DEBUG] vminit.bootstrap: pinging server
...

The problem needs to be first investigated in shared library and when that works, attempt integration in vminit.