search

lima-vm / lima

Linux virtual machines, with a focus on running containers
https://lima-vm.io/
Apache License 2.0
15.39k stars 604 forks source link

Support offline creation of machine instance #1422

Open afbjorklund opened 1 year ago

afbjorklund commented 1 year ago

Description

Currently the baseimage and the nerdctl-full can be cached, so that they don't have to be downloaded.

But then there are still some packages missing, which means that the machine needs Internet access.

It would be nice if there was a way to cache also these files, perhaps using something like repro-get ?

So that one could start up an instance without external network, after making sure everything is cached.

These are cached:

INFO[0001] Attempting to download the image from "https://cloud-images.ubuntu.com/releases/22.10/release-20230215/ubuntu-22.10-server-cloudimg-amd64.img"  digest="sha256:5e5c68cb12002111032d3239ade3763ce25639f1287a59d2509a1603c2b1f7e6"
INFO[0001] Using cache "/home/anders/.cache/lima/download/by-url-sha256/7db6503d427a306824b1f62ce99d3a19a4c83f3b69af39022477aecca5c69b33/data" 
INFO[0001] Attempting to download the nerdctl archive from "https://github.com/containerd/nerdctl/releases/download/v1.2.1/nerdctl-full-1.2.1-linux-amd64.tar.gz"  digest="sha256:e8a3e40d442c566ee494375a4c563121da69d7c7837f50f4a3a171742757b36c"
INFO[0002] Using cache "/home/anders/.cache/lima/download/by-url-sha256/3ca338b148379fea1376fb450380cb21a2b26d8b1618da6eb1c5c5e352b03ea7/data" 
~/.cache/lima/download/by-url-sha256
├── 3ca338b148379fea1376fb450380cb21a2b26d8b1618da6eb1c5c5e352b03ea7
│   ├── data
│   └── url
└── 7db6503d427a306824b1f62ce99d3a19a4c83f3b69af39022477aecca5c69b33
    ├── data
    ├── sha256.digest
    └── url

But these are not:

[   26.224771] cloud-init[2091]: + apt-get install -y --no-upgrade --no-install-recommends -q sshfs uidmap fuse3 dbus-user-session
[   26.261518] cloud-init[2091]: Reading package lists...
[   26.416159] cloud-init[2091]: Building dependency tree...
[   26.417192] cloud-init[2091]: Reading state information...
[   26.648233] cloud-init[2091]: Skipping fuse3, it is already installed and upgrade is not set.
[   26.648657] cloud-init[2091]: fuse3 set to manually installed.
[   26.648816] cloud-init[2091]: Skipping dbus-user-session, it is already installed and upgrade is not set.
[   26.648940] cloud-init[2091]: dbus-user-session set to manually installed.
[   26.649051] cloud-init[2091]: The following additional packages will be installed:
[   26.651759] cloud-init[2091]:   libsubid4
[   26.673312] cloud-init[2091]: The following NEW packages will be installed:
[   26.674092] cloud-init[2091]:   libsubid4 sshfs uidmap
[   27.802214] cloud-init[2091]: 0 upgraded, 3 newly installed, 0 to remove and 33 not upgraded.
[   27.803518] cloud-init[2091]: Need to get 91.3 kB of archives.
[   27.804323] cloud-init[2091]: After this operation, 438 kB of additional disk space will be used.
eb51492a6100940af250d65ad1c0d8f1fc9f9c645ebd4ad926da72ae3faf091a  pool/main/s/shadow/libsubid4_4.11.1+dfsg1-2ubuntu1.1_amd64.deb
8986b00a1b6547f013c0e6ab159fbc7379554a95d4d5da4b8359857c52650b5d  pool/universe/s/sshfs-fuse/sshfs_3.7.2-1_amd64.deb
217864fa6ae88b86af2179afa01c510021d51b61d5b16e3d3949dd4a4c4fd6f8  pool/main/s/shadow/uidmap_4.11.1+dfsg1-2ubuntu1.1_amd64.deb
afbjorklund commented 1 year ago

Maybe these could be integrated into the ~/.lima/cache ? And then mounted and used.

/var/cache/repro-get
├── blobs
│   └── sha256
│       ├── 217864fa6ae88b86af2179afa01c510021d51b61d5b16e3d3949dd4a4c4fd6f8
│       ├── 8986b00a1b6547f013c0e6ab159fbc7379554a95d4d5da4b8359857c52650b5d
│       └── eb51492a6100940af250d65ad1c0d8f1fc9f9c645ebd4ad926da72ae3faf091a
├── digests
│   └── by-url-sha256
│       ├── 18e1ea5f97f85270579d716eaba05a03313bd1707b322631503d64d74940fe08
│       ├── 87027974ef0c7f66c43c202c6f59850c6e7cbc449d8d5e9128a52bcaa13c23a1
│       └── c1698577bc5e6a2d0f92365749e47dae1dea093f9ae79ddedf5aa0193c24eb66
└── urls
    └── sha256
        ├── 217864fa6ae88b86af2179afa01c510021d51b61d5b16e3d3949dd4a4c4fd6f8
        ├── 8986b00a1b6547f013c0e6ab159fbc7379554a95d4d5da4b8359857c52650b5d
        └── eb51492a6100940af250d65ad1c0d8f1fc9f9c645ebd4ad926da72ae3faf091a

I want to be able to create a cache using a template, and then re-use (only) that offline.

An alternative would be to just mount the lists and archives, but that is a bigger download.

It seems like repro-get would be able to handle it, provided with an input file and a cache ?

afbjorklund commented 1 year ago

Importing the required commands as a library (to guestagent), might be less bloated....

        "github.com/reproducible-containers/repro-get/pkg/cache"
        "github.com/reproducible-containers/repro-get/pkg/distro"
        "github.com/reproducible-containers/repro-get/pkg/distro/alpine"
        "github.com/reproducible-containers/repro-get/pkg/distro/arch"
        "github.com/reproducible-containers/repro-get/pkg/distro/debian"
        "github.com/reproducible-containers/repro-get/pkg/distro/distroutil/detect"
        "github.com/reproducible-containers/repro-get/pkg/distro/fedora"
        "github.com/reproducible-containers/repro-get/pkg/distro/none"
        "github.com/reproducible-containers/repro-get/pkg/distro/ubuntu"
        "github.com/reproducible-containers/repro-get/pkg/downloader"
        "github.com/reproducible-containers/repro-get/pkg/filespec"
        "github.com/reproducible-containers/repro-get/pkg/envutil"

Like generate-hash and install-packages, something like that. More for cache* and downloader.

* i.e. it needs some code to translate back and forth between the different cache formats

afbjorklund commented 1 year ago

There was a surprisingly large number of packages installed for ubuntu, including a new kernel.

[   12.547536] cloud-init[2091]: + apt-get update
[   25.195512] cloud-init[2091]: + apt-get install -y --no-upgrade --no-install-recommends -q sshfs uidmap fuse3 dbus-user-session
82M /var/cache/apt
118M    /var/lib/apt
bd260f2cca262c1b76c8a28e56abba5d3ac51b04b78be1bd312c47ce7425318f  pool/main/c/curl/curl_7.85.0-1ubuntu0.3_amd64.deb
34aa4bed3bb64d8ea957b175ea3a1311230bbe97936415fc51ec11dfcd1c37b8  pool/main/f/fwupd-signed/fwupd-signed_1.51~22.10.1+1.2-3ubuntu0.2_amd64.deb
be20a9dd83c4de3e8cd9d0623469c483d81a055731bc6e4c57007b4fdefa15c8  pool/main/g/grub2-unsigned/grub-efi-amd64-bin_2.06-2ubuntu14.1_amd64.deb
7f0f79cbd7f6e9fd0a28aa3a7e5901ee51526adb7945fff0bb1c4a326f19e279  pool/main/g/grub2-signed/grub-efi-amd64-signed_1.187.3+2.06-2ubuntu14.1_amd64.deb
ddc05ad5df1e637719808e1cf58ddd9547284ac78cebc168605061b6a4fb60da  pool/main/c/curl/libcurl3-gnutls_7.85.0-1ubuntu0.3_amd64.deb
a9e8ea6902d25c7300127415261103aede5cdac63c299192439c724507700fa0  pool/main/c/curl/libcurl4_7.85.0-1ubuntu0.3_amd64.deb
05db4307a0b09b61095f25519af682673b18cb244389c02bb9bfdd66bb9e2465  pool/main/g/gnutls28/libgnutls30_3.7.7-2ubuntu2.1_amd64.deb
bc3dd5b29fce303db9278d7d8e53ff62e35ed8c67558c50c4cd0eaaf9da74d43  pool/main/o/openldap/libldap-2.5-0_2.5.14+dfsg-0ubuntu0.22.10.1_amd64.deb
75d67e788fd06e625a7e77e8970842e074c026914ef2e60e867d3f047b4fb633  pool/main/o/openldap/libldap-common_2.5.14+dfsg-0ubuntu0.22.10.1_all.deb
ba19ba35c32140f88157caeadd0d72a9dbd43a2fc2b97d6796ca9a7cbc753555  pool/main/n/netplan.io/libnetplan0_0.105-0ubuntu2.2_amd64.deb
ab3e2ee2d54ac58ef3796cac86af743112eda3f76d602b464e3193872edc80d5  pool/main/s/systemd/libnss-systemd_251.4-1ubuntu7.1_amd64.deb
fc12067c26fb69ea9130732ab0c7d491b90aaafd61b02425e69c5a061121146a  pool/main/n/nss/libnss3_3.82-1ubuntu0.1_amd64.deb
b99d9c79f6681ab40b83cf01ecb5c81112c2167712f44a4f7e35f8bc1ae07669  pool/main/s/systemd/libpam-systemd_251.4-1ubuntu7.1_amd64.deb
eb51492a6100940af250d65ad1c0d8f1fc9f9c645ebd4ad926da72ae3faf091a  pool/main/s/shadow/libsubid4_4.11.1+dfsg1-2ubuntu1.1_amd64.deb
55c840c00e1cac126be73ba145d8831c5c0318aa58a802be5d33b4a663d6063d  pool/main/s/systemd/libsystemd-shared_251.4-1ubuntu7.1_amd64.deb
187fa112cdc52a860efb4104e700d1deed8d2be888b2748fcddf0625b9655562  pool/main/s/systemd/libsystemd0_251.4-1ubuntu7.1_amd64.deb
45af912ba558c1bd6e956458ca02ae716b30c6ff0528078224dbf5e008a03335  pool/main/s/systemd/libudev1_251.4-1ubuntu7.1_amd64.deb
bcb2db01a67839bdd81c7c82f42595d850fac5f1c7bf42640c54b548f85d79ea  pool/main/l/linux-meta/linux-headers-generic_5.19.0.35.32_amd64.deb
bbecdd0fed1d71be1e4fece63305028c3323822038e80a99d59ee21f1269d9b1  pool/main/l/linux-meta/linux-headers-virtual_5.19.0.35.32_amd64.deb
6210d187d0cea0ae2271264e77c490eece7db97609d669f6354cec4f4a1a3940  pool/main/l/linux-meta/linux-image-virtual_5.19.0.35.32_amd64.deb
a09fcc9f23a22022f395a7b9ab2bf971550edf35b12c15017d2f1d5ef6c093cc  pool/main/l/linux-meta/linux-virtual_5.19.0.35.32_amd64.deb
99265d3b7bcc6a5200c7ae765cc55f32c5225f799cf9664db5744136dd417bee  pool/main/n/netplan.io/netplan.io_0.105-0ubuntu2.2_amd64.deb
b406ef3fc62743c300b4c5179fb45767f8ead124096337a679ce6d6f0ef9f9fd  pool/main/r/rsync/rsync_3.2.7-0ubuntu0.22.10.1_amd64.deb
e9dff8f490147e311e64d5f52fc6e62e927ce42d8689bb2df2ca8da7637a5592  pool/main/s/shim-signed/shim-signed_1.54+15.7-0ubuntu1_amd64.deb
8986b00a1b6547f013c0e6ab159fbc7379554a95d4d5da4b8359857c52650b5d  pool/universe/s/sshfs-fuse/sshfs_3.7.2-1_amd64.deb
2a08656031d10421bc4b3cedd6d6b952b1f47a4d291081c37e016c5d9ee21ace  pool/main/s/sudo/sudo_1.9.11p3-1ubuntu1.2_amd64.deb
906da7a4e740f65c7a26d2a5fc3790530dd4565874be8a13bebccd25e2659104  pool/main/s/systemd-hwe/systemd-hwe-hwdb_251.4.4_all.deb
1b439394732ee014ebc0292bc1881c900e9f4036985bb066be24de2e95ce4116  pool/main/s/systemd/systemd-resolved_251.4-1ubuntu7.1_amd64.deb
ab105cb2cc67f9eca4bbe8c3e746fcc675d6fdf24348900729f5a604526a9b9d  pool/main/s/systemd/systemd-sysv_251.4-1ubuntu7.1_amd64.deb
26662f6d4c2c99683d4c2661c98dfda2aea69954d6697bc02c8ff73095813640  pool/main/s/systemd/systemd-timesyncd_251.4-1ubuntu7.1_amd64.deb
3acf1e8df51bb9024db3588d22440aa8d72db77389b7f9dab096b5eb25e000c0  pool/main/s/systemd/systemd_251.4-1ubuntu7.1_amd64.deb
f9287c9119279bf6f9b603580a3ebd65de3a2ee65d0ec79e49b5380231e120db  pool/main/t/tar/tar_1.34+dfsg-1ubuntu0.1.22.10.1_amd64.deb
ddb0307b74a8d5efd21fcdde0170f4012b28d817f3c96dc693fc45662d70846b  pool/main/t/tcpdump/tcpdump_4.99.1-4ubuntu0.1_amd64.deb
72e63242b0784f27a263873ff3afa9ea0690459717e894372f7a37a83e7fac9b  pool/main/u/ubuntu-advantage-tools/ubuntu-advantage-tools_27.13.6~22.10.1_amd64.deb
daa19e0e620c04bb5f57db72c8088289259e9eef3aa0cc41f80d28a00732bb2f  pool/main/s/systemd/udev_251.4-1ubuntu7.1_amd64.deb
217864fa6ae88b86af2179afa01c510021d51b61d5b16e3d3949dd4a4c4fd6f8  pool/main/s/shadow/uidmap_4.11.1+dfsg1-2ubuntu1.1_amd64.deb

The package list for fedora (as used by podman), had a more expected content (from requirements)

[  139.673610] cloud-init[927]: + dnf install -y --setopt=install_weak_deps=False fuse-sshfs iptables fuse3
253M    /var/cache/dnf
4.2M    /var/lib/dnf
ebf5813d2f097c34901459e3b73ac82339e88b3678fc4ab72053f69e4dfa05ac  fuse3/3.10.5/5.fc37/x86_64/fuse3-libs-3.10.5-5.fc37.x86_64.rpm
5a3daece2b4e04d70e8abfc0414badd3fa5cde309a8a696cc9be3aa1095473a6  libnfnetlink/1.0.1/22.fc37/x86_64/libnfnetlink-1.0.1-22.fc37.x86_64.rpm
63ffa0b40c99006a9b4df1b63306b49a23678e6137a989c667f138a430a64803  libnetfilter_conntrack/1.0.8/5.fc37/x86_64/libnetfilter_conntrack-1.0.8-5.fc37.x86_64.rpm
71d528f369c4fb50f9d80b1f44b279671532b6850135094c511822a81a6ea498  rdma-core/41.0/1.fc37/x86_64/libibverbs-41.0-1.fc37.x86_64.rpm
b3906a4f8e7a640e356bfbd813bed5d40f2debbe4da822e6596f92eb75d03a28  libpcap/1.10.3/1.fc37/x86_64/libpcap-1.10.3-1.fc37.x86_64.rpm
1a16f7c9a161ff7c937d972f29d396d4c1621adf06e33d384e6d2b281cb9811c  iptables/1.8.8/4.fc37/x86_64/iptables-legacy-libs-1.8.8-4.fc37.x86_64.rpm
080697cd33c15d8b4732abdfd72e70b97534634571744d5c8be50cab9914b9a0  iptables/1.8.8/4.fc37/x86_64/iptables-libs-1.8.8-4.fc37.x86_64.rpm
79891a71896e6075dad3fe24704c85b80ed11fdd2d9f0111fb3e3b4a95c470c8  fuse3/3.10.5/5.fc37/x86_64/fuse-common-3.10.5-5.fc37.x86_64.rpm
9d7e2e7e2f8ac835319e99af2d38cc252fafd075c109f346ce6136023ec5027c  fuse3/3.10.5/5.fc37/x86_64/fuse3-3.10.5-5.fc37.x86_64.rpm
75e67e9ecf86d0bfb1a98c03e28f33b509605fac9cae859bc3681e63fffc59a9  fuse-sshfs/3.7.3/2.fc37/x86_64/fuse-sshfs-3.7.3-2.fc37.x86_64.rpm
e3785a00a67b7e54b5f98db551e6b641b2e3d2ecba123e7a4e94f621d9956513  iptables/1.8.8/4.fc37/x86_64/iptables-legacy-1.8.8-4.fc37.x86_64.rpm
afbjorklund commented 1 year ago

Somewhat related to non-packages, as well:

Currently it only supports "nerdctl-full" archives.

Starting kubernetes (k3s/k8s) also requires images:

185M k3s-airgap-images-amd64.tar.gz

208M k8s-airgap-images-amd64.tar.gz

afbjorklund commented 5 months ago

Doing a proof-of-concept using a regular HTTP proxy, for packages and for images.

It is opt-in, and only provides LIMA_CIDATA_HTTP_PROXY_LOCAL_PORT and proxy.crt

$ http_proxy=host.lima.internal:56778 curl http://example.com
2024/05/27 12:35:16 [009] INFO: Got request / example.com GET http://example.com/
2024/05/27 12:35:16 [009] INFO: Sending request GET http://example.com/
2024/05/27 12:35:16 [009] INFO: Received response 200 OK
2024/05/27 12:35:16 [009] INFO: Copying response to client 200 OK [200]
2024/05/27 12:35:16 [009] INFO: Copied 1256 bytes to client error=<nil>
$ https_proxy=host.lima.internal:56778 curl --cacert proxy.crt --head https://example.com
2024/05/27 12:35:42 [011] INFO: Running 2 CONNECT handlers
2024/05/27 12:35:42 [011] INFO: on 0th handler: &{2 <nil> 0xeb6260} example.com:443
2024/05/27 12:35:42 [011] INFO: Assuming CONNECT is TLS, mitm proxying it
2024/05/27 12:35:42 [011] INFO: signing for example.com
2024/05/27 12:35:42 [011] WARN: Cannot handshake client example.com:443 EOF
2024/05/27 12:35:45 [012] INFO: Running 2 CONNECT handlers
2024/05/27 12:35:45 [012] INFO: on 0th handler: &{2 <nil> 0xeb6260} example.com:443
2024/05/27 12:35:45 [012] INFO: Assuming CONNECT is TLS, mitm proxying it
2024/05/27 12:35:45 [012] INFO: signing for example.com
2024/05/27 12:35:45 [013] INFO: req example.com:443
2024/05/27 12:35:45 [013] INFO: Sending request HEAD https://example.com:443/
2024/05/27 12:35:46 [013] INFO: resp 200 OK
2024/05/27 12:35:46 [012] INFO: Exiting on EOF