Replace hyperkit with lima

[jandubois]

Once lima is included in RD (#379), it should replace hyperkit.

Files can either be used via mounting (it doesn't require root and goes over ssh) in the config file, or files can be copied in via ssh (user name is the same as host user, port is specified in the machine config):

$ scp -P 60020 boot2tcl-1.1.1.iso jan@127.0.0.1:/tmp/image.iso
boot2tcl-1.1.1.iso                                                                        100%   52MB 171.5MB/s   00:00

You can run commands via ssh:

$ limactl shell alpine uname -a
Linux lima-alpine 5.10.38-0-virt #1-Alpine SMP Fri, 21 May 2021 11:41:13 UTC x86_64 Linux

Note that the ISO has a /bin/bash file for internal lima reasons, but it really is just forwarding to /bin/ash.

Status of the machine is available via ls:

$ limactl ls --json
{"name":"alpine","status":"Running","dir":"/Users/jan/.lima/alpine","arch":"x86_64","sshLocalPort":60020,"hostAgentPID":77846,"qemuPID":77847}

[mook-as]

Trying the instructions from https://github.com/rancher-sandbox/rancher-desktop/issues/379#issue-928676490 leads to:

INFO[0000] Using the existing instance "alpine"         
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "…/serial.log") 
INFO[0000] SSH Local Port: 60020                        
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh" 
⋮
INFO[0592] [hostagent] Waiting for the essential requirement 1 of 2: "ssh" 
INFO[0592] [hostagent] Waiting for the essential requirement 2 of 2: "the guest agent to be running" 
FATA[0600] did not receive an event with the "running" status 

I think the issue here is that sshd take so long to start, and there's a total bound of 10 minutes (!) for the whole process, that it aborts… Note that, watching the serial console, it gets to lima-alpine login: (i.e. after cloud-init) within about 10 seconds. sshd is up soon after that (as in, I couldn't figure out the delay). However, I do see lots of errors in /var/log/messages:

Jul  6 00:26:23 lima-alpine auth.info sshd[2722]: Connection closed by authenticating user mookas 192.168.5.2 port 50730 [preauth]

Irrelevant side note: If I understand the naming scheme correctly, qemu-system-x86_64 would mean we're not using the CPU virtualization support (VT-x)?

[jandubois]

Something is wrong; starting alpine with lima takes 20s on my setup:

$ time make lima
ALPINE_VERSION=3.13.5 EDITION=std ./lima.sh
WARN[0000] Ignoring non-existent instance "std"
INFO[0000] Terminal is not available, proceeding without opening an editor
INFO[0000] Attempting to download the image from "/Users/jan/suse/alpine-lima/iso/alpine-lima-std-3.13.5-x86_64.iso"
INFO[0000] Downloaded image from "/Users/jan/suse/alpine-lima/iso/alpine-lima-std-3.13.5-x86_64.iso"
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/jan/.lima/std/serial.log")
INFO[0000] SSH Local Port: 40022
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0020] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0020] [hostagent] The essential requirement 1 of 4 is satisfied
INFO[0020] [hostagent] Waiting for the essential requirement 2 of 4: "sshfs binary to be installed"
INFO[0020] [hostagent] The essential requirement 2 of 4 is satisfied
INFO[0020] [hostagent] Waiting for the essential requirement 3 of 4: "/etc/fuse.conf to contain \"user_allow_other\""
INFO[0020] [hostagent] The essential requirement 3 of 4 is satisfied
INFO[0020] [hostagent] Waiting for the essential requirement 4 of 4: "the guest agent to be running"
INFO[0020] [hostagent] The essential requirement 4 of 4 is satisfied
INFO[0020] [hostagent] Mounting "/Users/jan"
INFO[0020] READY. Run `limactl shell std` to open the shell.

real    0m20.491s

You can edit the VM config to include a real console (which will slow the VM down more than expected, but it is fine for debugging):

video:
  display: cocoa

You should be able to login within 5s as root (without password). Maybe you can figure out from inside what is going wrong.

I wonder if something is going wrong with your ~/.ssh/authorized_keys inside the VM. lima will install all ~/.ssh/*.pub keys from the host into the VM to allow you to authenticate with any of them. It also doesn't specify the user to the ssh command, so if you have a host * section in your ~/.ssh/config, you may want to take a look (both issues will be addressed in Lima 0.5.0).

If I understand the naming scheme correctly, qemu-system-x86_64 would mean we're not using the CPU virtualization support (VT-x)?

The file should be signed with the "hypervisor framework" entitlement and should use CPU virtualization support (via the -machine q35,accel=hvf option to qemu).

I don't think it matters that the qemu binary has an ad-hoc signature from a different machine; I think your problem is related to ssh configuration. But just in case you can try to resign the binary on your machine (adjust path to match):

codesign --sign - --force --preserve-metadata=entitlements /usr/local/bin/qemu-system-x86_64

[jandubois]

I wonder if something is going wrong with your ~/.ssh/authorized_keys inside the VM.

@mook-as If this is indeed the case, please preserve the files causing the problem, including the public key on your host, and also the /var/lib/cloud/instance/user-data.txt from inside the guest. It should have a section like this:

users:
  - name: "jan"
    uid: "501"
    homedir: "/home/jan.linux"
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    lock_passwd: true
    ssh-authorized-keys:
      - ssh-ed25519 AAAAC3NzaC1lZDI1... comment
      - ssh-rsa AAAAB3NzaC1yc2E...== comment

Let me know if anything there is malformed!

[jandubois]

401 bumps lima to v0.5.0.

Relevant changes for Rancher Desktop:

• LIMA_HOME can be used to create a private ~/.lima directory separate from user instances.
• The config file can disable re-use of ~/.ssh/*.pub files (lima now always adds its own identity from $LIMA_HOME/_config/user.pub):

  ssh:
    loadDotSSHPubKeys: false

• New limactl cp command can be used to copy files between host and guest instances, e.g. limactl cp default:/etc/hostname . This allows RD to use an empty mounts: [] section.