openllb / hlb

A developer-first language to build and test any software efficiently
https://openllb.github.io/hlb/
Apache License 2.0
108 stars 12 forks source link

Add support for stargz images #307

Closed hinshun closed 2 years ago

hinshun commented 2 years ago

See: https://github.com/moby/buildkit/blob/master/docs/stargz-estargz.md

Example manifest:

{
   "mediaType": "application/vnd.oci.image.manifest.v1+json",
   "schemaVersion": 2,
   "config": {
      "mediaType": "application/vnd.oci.image.config.v1+json",
      "digest": "sha256:cba6a63f22c24075f2f0c9efeea9603dc9845aa3afcec9cd061c08f5d12ecc75",
      "size": 1031
   },
   "layers": [
      {
         "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
         "digest": "sha256:25af8f56fba7da444a7be3ff6a6df9ad9c15b19c624e1c2338bb0d1b1b226f69",
         "size": 314668980
      },
      {
         "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
         "digest": "sha256:7c084638d9c588cc1efc79c73a44e1424899eb10b4067119393a0dfab8400876",
         "size": 485,
         "annotations": {
            "containerd.io/snapshot/stargz/toc.digest": "sha256:c859b4593016fd4de87ba9dd20076c800428dc2f4b3b8b5e2b3ba1dc5d25f6d3",
            "io.containers.estargz.uncompressed-size": "4096"
         }
      }
   ]
}
hinshun commented 2 years ago

Pending discussion in: https://github.com/moby/buildkit/issues/2663

hinshun commented 2 years ago

BuildKit's stargz compression default behavior is to leave standard layers alone, creating a mixed mode image where some layers are standard and some are stargz. I believe this is non-intuitive for users, so in the spirit of HLB, we should always end up with a fully stargz image, while having options available for mixed mode later.

See: https://github.com/moby/buildkit/issues/2663#issuecomment-1047300669

hinshun commented 2 years ago

Also it seems that gateway exec unlazy stargz image, looking at our gateway code, nothing sticks out on our usage.

See: https://github.com/moby/buildkit/issues/2664

hinshun commented 2 years ago

Added tests and disabled for moby engine until it supports stargz.