search

lima-vm / lima

Linux virtual machines, with a focus on running containers
https://lima-vm.io/
Apache License 2.0
14.55k stars 570 forks source link

IPFS for downloading images, using decentralized peer-to-peer #2407

Open afbjorklund opened 3 weeks ago

afbjorklund commented 3 weeks ago

Description

It should be possible to upload the images to IPFS, and then use such an address to download them p2p (by cid):

ipfs://bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui/ubuntu-24.04-server-cloudimg-amd64.img

CLI: https://docs.ipfs.tech/how-to/kubo-basic-cli/#install-kubo

GUI: https://docs.ipfs.tech/how-to/desktop-app/#install-ipfs-desktop

Implementation itself should be similar to ORAS: #2405

Setting up a hosting network is a completely different story.

Using an external ipfs tool on purpose, to cut down on dependencies.

It would be possible to use github.com/ipfs/kubo directly, as a "library".

afbjorklund commented 1 week ago

Best case scenario, where both images and archives are found in local diskstore:

? Creating an instance "ipfs" Proceed with the current configuration
INFO[0000] Attempting to download the image              arch=x86_64 digest="sha256:32a9d30d18803da72f5936cf2b7b9efcb4d0bb63c67933f17e3bdfd1751de3f3" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240423/ubuntu-24.04-server-cloudimg-amd64.img"
Downloading the image (ubuntu-24.04-server-cloudimg-amd64.img)
453.00 MiB / 453.00 MiB [---------------------------------] 100.00% 340.48 MiB/s
INFO[0002] Used ipfs "bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui" 
INFO[0002] Attempting to download the nerdctl archive    arch=x86_64 digest="sha256:2c841e097fcfb5a1760bd354b3778cb695b44cd01f9f271c17507dc4a0b25606" location="https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-amd64.tar.gz"
Downloading the nerdctl archive (nerdctl-full-1.7.6-linux-amd64.tar.gz)
226.46 MiB / 226.46 MiB [---------------------------------] 100.00% 360.66 MiB/s
INFO[0004] Used ipfs "bafybeieipdaxd3fzy3j7syzzxdaqxramk65j7ajzcqgmi6b5jyq4jgbwue" 
INFO[0004] Run `limactl start ipfs` to start the instance.

When ipfs (kubo) is not available, or image not found in IPFS, it would use HTTP...

In either case, the digest is verified before putting in the cache (with the url as key)

afbjorklund commented 1 week ago

It is still possible to use ipfs:// as the primary, but now using a secondary cid:

images:
# Try to use release-yyyyMMdd image if available. Note that release-yyyyMMdd will be removed after several months.
- location: "https://cloud-images.ubuntu.com/releases/24.04/release-20240423/ubuntu-24.04-server-cloudimg-amd64.img"
  arch: "x86_64"
  digest: "sha256:32a9d30d18803da72f5936cf2b7b9efcb4d0bb63c67933f17e3bdfd1751de3f3"
  cid: bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui
- location: "https://cloud-images.ubuntu.com/releases/24.04/release-20240423/ubuntu-24.04-server-cloudimg-arm64.img"
  arch: "aarch64"
  digest: "sha256:c841bac00925d3e6892d979798103a867931f255f28fefd9d5e07e3e22d0ef22"
  cid: bafybeidyzm3zkljztw3xqqjjapcqzsr4odz4evnmk4kfhdmmgird6citbi
# Fallback to the latest release image.
# Hint: run `limactl prune` to invalidate the cache
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.img"
  arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img"
  arch: "aarch64"
archives:
- location: "https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-amd64.tar.gz"
  arch: "x86_64"
  digest: "sha256:2c841e097fcfb5a1760bd354b3778cb695b44cd01f9f271c17507dc4a0b25606"
  cid: bafybeieipdaxd3fzy3j7syzzxdaqxramk65j7ajzcqgmi6b5jyq4jgbwue
- location: "https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-arm64.tar.gz"
  arch: "aarch64"
  digest: "sha256:77c747f09853ee3d229d77e8de0dd3c85622537d82be57433dc1fca4493bab95"
  cid: bafybeibptqoynryoxytxffgyuqjnmwpuujwp6wlps4spygapuwphzkeieq
# No arm-v7
# No riscv64

That means that it can share the digest (and the cache) with the https:// downloads.