search

edgelesssys / contrast

Deploy and manage confidential containers on Kubernetes
https://docs.edgeless.systems/contrast
GNU Affero General Public License v3.0
161 stars 6 forks source link

generate: `Failed to pull container image manifest and config`, 504, `Unable to locate the requested resource` #516

Closed blenessy closed 3 weeks ago

blenessy commented 1 month ago

I am testing contrast for the first time. I finished the Getting Started steps successfully, now I'm working myself through the Confidential emoji voting example.

When I run the following command for the first time, the contrast cli crashes: 

$ contrast generate deployment/
...
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body is content-length (73 bytes)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] pooling idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Making authentication call realm="https://ghcr.io/token" service=Some("ghcr.io") scope="repository:3u13r/emojivoto-web:pull"
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] reuse idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] flushed 115 bytes
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] parsed 5 headers
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body is content-length (69 bytes)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] pooling idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Received response from auth request: {"token":"djE6M3UxM3IvZW1vaml2b3RvLXdlYjoxNzE3MjMyMzg5MDkwMDU4OTA5"}

[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Successfully authorized for image 'Reference { registry: "ghcr.io", repository: "3u13r/emojivoto-web", tag: Some("coco-1"), digest: None }'
[2024-06-01T08:59:49Z DEBUG oci_distribution::token_cache] Cannot extract expiration from token's claims, assuming a 60 seconds validity token=Bearer(Token { token: "<redacted>" })
[2024-06-01T08:59:49Z DEBUG oci_distribution::token_cache] Inserting token registry=ghcr.io repository=3u13r/emojivoto-web op=Pull expiration=1717232449
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Pulling image manifest from https://ghcr.io/v2/3u13r/emojivoto-web/manifests/coco-1
[2024-06-01T08:59:49Z DEBUG oci_distribution::token_cache] Fetching token registry=ghcr.io repository=3u13r/emojivoto-web op=Pull expiration=1717232449 miss=false expired=false
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Using bearer token authentication.
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] reuse idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] flushed 358 bytes
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] parsed 7 headers
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body is content-length (953 bytes)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] pooling idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] validating manifest: {
       "schemaVersion": 2,
       "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
       "config": {
          "mediaType": "application/vnd.docker.container.image.v1+json",
          "size": 1778,
          "digest": "sha256:301126f7d829054af8d3fcc70fe4868b35a4014b16d302ae30276f473a3c0b62"
       },
       "layers": [
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "size": 49582225,
             "digest": "sha256:90e5e7d8b87a34877f61c2b86d053db1c4f440b9054cf49573e3be5d6a674a47"
          },
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "size": 42393741,
             "digest": "sha256:c387947a6ad61575d4051e0758631f3143e3083f683ff2228f96c80c03a91947"
          },
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "size": 12893434,
             "digest": "sha256:b3235cb48fb830f8c3d1f243100698c3bcc2e0f35e424748d8249e8cb8957bd3"
          }
       ]
    }
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Parsing response as Manifest: {
       "schemaVersion": 2,
       "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
       "config": {
          "mediaType": "application/vnd.docker.container.image.v1+json",
          "size": 1778,
          "digest": "sha256:301126f7d829054af8d3fcc70fe4868b35a4014b16d302ae30276f473a3c0b62"
       },
       "layers": [
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "size": 49582225,
             "digest": "sha256:90e5e7d8b87a34877f61c2b86d053db1c4f440b9054cf49573e3be5d6a674a47"
          },
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "size": 42393741,
             "digest": "sha256:c387947a6ad61575d4051e0758631f3143e3083f683ff2228f96c80c03a91947"
          },
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "size": 12893434,
             "digest": "sha256:b3235cb48fb830f8c3d1f243100698c3bcc2e0f35e424748d8249e8cb8957bd3"
          }
       ]
    }
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Pulling config layer
[2024-06-01T08:59:49Z DEBUG oci_distribution::token_cache] Fetching token registry=ghcr.io repository=3u13r/emojivoto-web op=Pull expiration=1717232449 miss=false expired=false
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Using bearer token authentication.
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] reuse idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] flushed 419 bytes
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] parsed 6 headers
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body is empty
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] pooling idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG reqwest::async_impl::client] redirecting 'https://ghcr.io/v2/3u13r/emojivoto-web/blobs/sha256:301126f7d829054af8d3fcc70fe4868b35a4014b16d302ae30276f473a3c0b62' to 'https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:301126f7d829054af8d3fcc70fe4868b35a4014b16d302ae30276f473a3c0b62?se=2024-06-01T09%3A05%3A00Z&sig=C76R0MrO%2Ba%2FIXWAZECTIsGS0ZC8wGNv8g04YJ8kDtL8%3D&sp=r&spr=https&sr=b&sv=2019-12-12'
[2024-06-01T08:59:49Z DEBUG reqwest::connect] starting new connection: https://pkg-containers.githubusercontent.com/
[2024-06-01T08:59:49Z DEBUG hyper::client::connect::dns] resolving host="pkg-containers.githubusercontent.com"
[2024-06-01T08:59:49Z DEBUG hyper::client::connect::http] connecting to 185.199.111.154:443
[2024-06-01T08:59:49Z DEBUG hyper::client::connect::http] connected to 185.199.111.154:443
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] flushed 595 bytes
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] parsed 27 headers
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body is content-length (1778 bytes)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] pooling idle connection for ("https", pkg-containers.githubusercontent.com)
[2024-06-01T08:59:49Z DEBUG genpolicy::registry] digest_hash: "sha256:0fd9bf6f7dcb99bdb076144546b663ba6c3eb457cbb48c1d3fceb591d207289c"
[2024-06-01T08:59:49Z DEBUG genpolicy::registry] manifest: {
      "schemaVersion": 2,
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "digest": "sha256:301126f7d829054af8d3fcc70fe4868b35a4014b16d302ae30276f473a3c0b62",
        "size": 1778
      },
      "layers": [
        {
          "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
          "digest": "sha256:90e5e7d8b87a34877f61c2b86d053db1c4f440b9054cf49573e3be5d6a674a47",
          "size": 49582225
        },
        {
          "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
          "digest": "sha256:c387947a6ad61575d4051e0758631f3143e3083f683ff2228f96c80c03a91947",
          "size": 42393741
        },
        {
          "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
          "digest": "sha256:b3235cb48fb830f8c3d1f243100698c3bcc2e0f35e424748d8249e8cb8957bd3",
          "size": 12893434
        }
      ]
    }
{
  "architecture": "amd64",
  "config": {
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "SVC_NAME=emojivoto-web"
    ],
    "Entrypoint": [
      "/bin/sh",
      "-c",
      "cd /usr/local/bin && $SVC_NAME"
    ],
    "OnBuild": null
  },
  "created": "2023-12-19T11:54:11.037356334+01:00",
  "history": [
    {
      "created": "2023-11-21T05:21:24.536066751Z",
      "created_by": "/bin/sh -c #(nop) ADD file:39d17d28c5de0bd629e5b7c8190228e5a445d61d668e189b7523e90e68f78244 in / "
    },
    {
      "created": "2023-11-21T05:21:25.128983079Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
      "empty_layer": true
    },
    {
      "created": "2023-12-18T16:54:05.180251548+01:00",
      "created_by": "RUN /bin/sh -c apt-get update     && apt-get install -y --no-install-recommends         curl         dnsutils         iptables         jq         nghttp2     && rm -rf /var/lib/apt/lists/* # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-12-19T11:54:11.037356334+01:00",
      "created_by": "ARG svc_name",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-12-19T11:54:11.037356334+01:00",
      "created_by": "COPY emojivoto-web/target/ /usr/local/bin/ # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-12-19T11:54:11.037356334+01:00",
      "created_by": "ENV SVC_NAME=emojivoto-web",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-12-19T11:54:11.037356334+01:00",
      "created_by": "ENTRYPOINT [\"/bin/sh\" \"-c\" \"cd /usr/local/bin && $SVC_NAME\"]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:7cea17427f83f6c4706c74f94fb6d7925b06ea9a0701234f1a9d43f6af11432a",
      "sha256:26f0b7aa91062941f40aa2484cee756129297e268ed78c03f2d33a17f15babe5",
      "sha256:479d2650b89c1ccac225a262c763c1aaf2c24db7f553feca71d3ee5c137ac02f"
    ]
  }
}[2024-06-01T08:59:49Z INFO  genpolicy::registry] Using cache file
[2024-06-01T08:59:49Z INFO  genpolicy::registry] dm-verity root hash: 
[2024-06-01T08:59:49Z INFO  genpolicy::registry] Pulling layer "sha256:90e5e7d8b87a34877f61c2b86d053db1c4f440b9054cf49573e3be5d6a674a47"
[2024-06-01T08:59:49Z DEBUG oci_distribution::token_cache] Fetching token registry=ghcr.io repository=3u13r/emojivoto-web op=Pull expiration=1717232449 miss=false expired=false
[2024-06-01T08:59:49Z DEBUG oci_distribution::client] Using bearer token authentication.
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] reuse idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] flushed 419 bytes
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] parsed 6 headers
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::conn] incoming body is empty
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] pooling idle connection for ("https", ghcr.io)
[2024-06-01T08:59:49Z DEBUG reqwest::async_impl::client] redirecting 'https://ghcr.io/v2/3u13r/emojivoto-web/blobs/sha256:90e5e7d8b87a34877f61c2b86d053db1c4f440b9054cf49573e3be5d6a674a47' to 'https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:90e5e7d8b87a34877f61c2b86d053db1c4f440b9054cf49573e3be5d6a674a47?se=2024-06-01T09%3A05%3A00Z&sig=9YGK8YS2mXEeaiRKzTzoqBrpB1AgzWiOI1t7iizaKGQ%3D&sp=r&spr=https&sr=b&sv=2019-12-12'
[2024-06-01T08:59:49Z DEBUG hyper::client::pool] reuse idle connection for ("https", pkg-containers.githubusercontent.com)
[2024-06-01T08:59:49Z DEBUG hyper::proto::h1::io] flushed 591 bytes
[2024-06-01T08:59:50Z DEBUG hyper::proto::h1::io] parsed 27 headers
[2024-06-01T08:59:50Z DEBUG hyper::proto::h1::conn] incoming body is content-length (49582225 bytes)
[2024-06-01T08:59:52Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T08:59:52Z DEBUG hyper::client::pool] pooling idle connection for ("https", pkg-containers.githubusercontent.com)
[2024-06-01T08:59:52Z INFO  genpolicy::registry] Decompressing layer
[2024-06-01T08:59:58Z INFO  genpolicy::registry] Adding tarfs index to layer
[2024-06-01T08:59:59Z INFO  genpolicy::registry] Calculating dm-verity root hash
[2024-06-01T09:00:02Z INFO  genpolicy::registry] dm-verity root hash: 652402d4533346ab35768af3748836beacbbbff5b27d528ddf28f78cc6415c3a
[2024-06-01T09:00:02Z INFO  genpolicy::registry] Using cache file
[2024-06-01T09:00:02Z INFO  genpolicy::registry] dm-verity root hash: 
[2024-06-01T09:00:02Z INFO  genpolicy::registry] Pulling layer "sha256:c387947a6ad61575d4051e0758631f3143e3083f683ff2228f96c80c03a91947"
[2024-06-01T09:00:02Z DEBUG oci_distribution::token_cache] Fetching token registry=ghcr.io repository=3u13r/emojivoto-web op=Pull expiration=1717232449 miss=false expired=false
[2024-06-01T09:00:02Z DEBUG oci_distribution::client] Using bearer token authentication.
[2024-06-01T09:00:02Z DEBUG hyper::client::pool] reuse idle connection for ("https", ghcr.io)
[2024-06-01T09:00:02Z DEBUG hyper::proto::h1::io] flushed 419 bytes
[2024-06-01T09:00:02Z DEBUG hyper::proto::h1::io] parsed 6 headers
[2024-06-01T09:00:02Z DEBUG hyper::proto::h1::conn] incoming body is empty
[2024-06-01T09:00:02Z DEBUG hyper::client::pool] pooling idle connection for ("https", ghcr.io)
[2024-06-01T09:00:02Z DEBUG reqwest::async_impl::client] redirecting 'https://ghcr.io/v2/3u13r/emojivoto-web/blobs/sha256:c387947a6ad61575d4051e0758631f3143e3083f683ff2228f96c80c03a91947' to 'https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:c387947a6ad61575d4051e0758631f3143e3083f683ff2228f96c80c03a91947?se=2024-06-01T09%3A10%3A00Z&sig=BW2WhI4iQprtFM8SBgDhghzzZIeubwCDv1AqLNHYjoI%3D&sp=r&spr=https&sr=b&sv=2019-12-12'
[2024-06-01T09:00:02Z DEBUG hyper::client::pool] reuse idle connection for ("https", pkg-containers.githubusercontent.com)
[2024-06-01T09:00:02Z DEBUG hyper::proto::h1::io] flushed 591 bytes
[2024-06-01T09:00:03Z DEBUG hyper::proto::h1::io] parsed 27 headers
[2024-06-01T09:00:03Z DEBUG hyper::proto::h1::conn] incoming body is content-length (42393741 bytes)
[2024-06-01T09:00:04Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:04Z DEBUG hyper::client::pool] pooling idle connection for ("https", pkg-containers.githubusercontent.com)
[2024-06-01T09:00:04Z INFO  genpolicy::registry] Decompressing layer
[2024-06-01T09:00:09Z INFO  genpolicy::registry] Adding tarfs index to layer
[2024-06-01T09:00:11Z INFO  genpolicy::registry] Calculating dm-verity root hash
[2024-06-01T09:00:13Z INFO  genpolicy::registry] dm-verity root hash: 616bca46f1661535a8fc9280a2cf5c82d5a6d1819fc20328b28eacb942a4b8fd
[2024-06-01T09:00:13Z INFO  genpolicy::registry] Using cache file
[2024-06-01T09:00:13Z INFO  genpolicy::registry] dm-verity root hash: 
[2024-06-01T09:00:13Z INFO  genpolicy::registry] Pulling layer "sha256:b3235cb48fb830f8c3d1f243100698c3bcc2e0f35e424748d8249e8cb8957bd3"
[2024-06-01T09:00:13Z DEBUG oci_distribution::token_cache] Fetching token registry=ghcr.io repository=3u13r/emojivoto-web op=Pull expiration=1717232449 miss=false expired=false
[2024-06-01T09:00:13Z DEBUG oci_distribution::client] Using bearer token authentication.
[2024-06-01T09:00:13Z DEBUG hyper::client::pool] reuse idle connection for ("https", ghcr.io)
[2024-06-01T09:00:13Z DEBUG hyper::proto::h1::io] flushed 419 bytes
[2024-06-01T09:00:13Z DEBUG hyper::proto::h1::io] parsed 6 headers
[2024-06-01T09:00:13Z DEBUG hyper::proto::h1::conn] incoming body is empty
[2024-06-01T09:00:13Z DEBUG hyper::client::pool] pooling idle connection for ("https", ghcr.io)
[2024-06-01T09:00:13Z DEBUG reqwest::async_impl::client] redirecting 'https://ghcr.io/v2/3u13r/emojivoto-web/blobs/sha256:b3235cb48fb830f8c3d1f243100698c3bcc2e0f35e424748d8249e8cb8957bd3' to 'https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:b3235cb48fb830f8c3d1f243100698c3bcc2e0f35e424748d8249e8cb8957bd3?se=2024-06-01T09%3A10%3A00Z&sig=m4WzkNcxTLx6KQA0FPV33qF8mGtRu0woiTEc4lxiCj4%3D&sp=r&spr=https&sr=b&sv=2019-12-12'
[2024-06-01T09:00:13Z DEBUG hyper::client::pool] reuse idle connection for ("https", pkg-containers.githubusercontent.com)
[2024-06-01T09:00:13Z DEBUG hyper::proto::h1::io] flushed 591 bytes
[2024-06-01T09:00:13Z DEBUG hyper::proto::h1::io] parsed 27 headers
[2024-06-01T09:00:13Z DEBUG hyper::proto::h1::conn] incoming body is content-length (12893434 bytes)
[2024-06-01T09:00:14Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:14Z DEBUG hyper::client::pool] pooling idle connection for ("https", pkg-containers.githubusercontent.com)
[2024-06-01T09:00:14Z INFO  genpolicy::registry] Decompressing layer
[2024-06-01T09:00:15Z INFO  genpolicy::registry] Adding tarfs index to layer
[2024-06-01T09:00:15Z INFO  genpolicy::registry] Calculating dm-verity root hash
[2024-06-01T09:00:16Z INFO  genpolicy::registry] dm-verity root hash: f4f4487c9d92c464ce43bb605724c9a84c413d78433344147d49b586f2a05b8b
[2024-06-01T09:00:16Z DEBUG genpolicy::pod] Adding pause container...
[2024-06-01T09:00:16Z INFO  genpolicy::registry] ============================================
[2024-06-01T09:00:16Z INFO  genpolicy::registry] Pulling manifest and config for "mcr.microsoft.com/oss/kubernetes/pause:3.6"
[2024-06-01T09:00:16Z DEBUG genpolicy::registry] build_auth: Reference { registry: "mcr.microsoft.com", repository: "oss/kubernetes/pause", tag: Some("3.6"), digest: None }
[2024-06-01T09:00:16Z DEBUG genpolicy::registry] build_auth: Docker config not found - using anonymous access.
[2024-06-01T09:00:16Z DEBUG oci_distribution::token_cache] Fetching token registry=mcr.microsoft.com repository=oss/kubernetes/pause op=Pull miss=true
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Authorizing for image: Reference { registry: "mcr.microsoft.com", repository: "oss/kubernetes/pause", tag: Some("3.6"), digest: None }
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] url="https://mcr.microsoft.com/v2/"
[2024-06-01T09:00:16Z DEBUG reqwest::connect] starting new connection: https://mcr.microsoft.com/
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::dns] resolving host="mcr.microsoft.com"
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::http] connecting to 204.79.197.219:443
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::http] connected to 204.79.197.219:443
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] flushed 59 bytes
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] parsed 16 headers
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body is content-length (2 bytes)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] pooling idle connection for ("https", mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Pulling image manifest from https://mcr.microsoft.com/v2/oss/kubernetes/pause/manifests/3.6
[2024-06-01T09:00:16Z DEBUG oci_distribution::token_cache] Fetching token registry=mcr.microsoft.com repository=oss/kubernetes/pause op=Pull miss=true
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] reuse idle connection for ("https", mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] flushed 286 bytes
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] parsed 20 headers
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body is content-length (2876 bytes)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] pooling idle connection for ("https", mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] validating manifest: {
       "schemaVersion": 2,
       "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
       "manifests": [
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 526,
             "digest": "sha256:dcc1da6e41b612a7f7c084d7b74ec4217628481653bd312245fd482669d8a1c3",
             "platform": {
                "architecture": "amd64",
                "os": "linux"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 526,
             "digest": "sha256:f73f588eec188ecccbabaff4c1cc56861c6e1505b902d98326c6a0e4e12f1c42",
             "platform": {
                "architecture": "arm64",
                "os": "linux"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:f4ac294d5e0e65f28e2aaee8fb78344b853583e8798c80de01da424f8840f93f",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.17763.2300"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:0b7090046c58097b8b19e32f17e892cc54b62e0c2c098c140fc1bca38227adf7",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.18362.1256"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:51eac0aa41cb49323d008e33e4d25f9aabae863de5e600e49ad059fcf863fce8",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.18363.1556"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:c6bce28e0e0e416014bc2ef43b3703d57484a63d18938a9fcb1191fc6f7bf18e",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.19041.1348"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:9e55b7e4797103b19b84d2d1d91be17cd1ab4a1d4501b05adc28045f97f63997",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.19042.1348"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:f790758fa500f4abe4e2e8fcfd86e71d1f99b81195e7c4ad62777b20892c0129",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.20348.350"
             }
          }
       ]
    }
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Parsing response as Manifest: {
       "schemaVersion": 2,
       "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
       "manifests": [
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 526,
             "digest": "sha256:dcc1da6e41b612a7f7c084d7b74ec4217628481653bd312245fd482669d8a1c3",
             "platform": {
                "architecture": "amd64",
                "os": "linux"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 526,
             "digest": "sha256:f73f588eec188ecccbabaff4c1cc56861c6e1505b902d98326c6a0e4e12f1c42",
             "platform": {
                "architecture": "arm64",
                "os": "linux"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:f4ac294d5e0e65f28e2aaee8fb78344b853583e8798c80de01da424f8840f93f",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.17763.2300"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:0b7090046c58097b8b19e32f17e892cc54b62e0c2c098c140fc1bca38227adf7",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.18362.1256"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:51eac0aa41cb49323d008e33e4d25f9aabae863de5e600e49ad059fcf863fce8",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.18363.1556"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:c6bce28e0e0e416014bc2ef43b3703d57484a63d18938a9fcb1191fc6f7bf18e",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.19041.1348"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:9e55b7e4797103b19b84d2d1d91be17cd1ab4a1d4501b05adc28045f97f63997",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.19042.1348"
             }
          },
          {
             "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
             "size": 1157,
             "digest": "sha256:f790758fa500f4abe4e2e8fcfd86e71d1f99b81195e7c4ad62777b20892c0129",
             "platform": {
                "architecture": "amd64",
                "os": "windows",
                "os.version": "10.0.20348.350"
             }
          }
       ]
    }
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Inspecting Image Index Manifest
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Selected manifest entry with digest: sha256:dcc1da6e41b612a7f7c084d7b74ec4217628481653bd312245fd482669d8a1c3
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Pulling image manifest from https://mcr.microsoft.com/v2/oss/kubernetes/pause/manifests/sha256:dcc1da6e41b612a7f7c084d7b74ec4217628481653bd312245fd482669d8a1c3
[2024-06-01T09:00:16Z DEBUG oci_distribution::token_cache] Fetching token registry=mcr.microsoft.com repository=oss/kubernetes/pause op=Pull miss=true
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] reuse idle connection for ("https", mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] flushed 354 bytes
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] parsed 20 headers
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body is content-length (526 bytes)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] pooling idle connection for ("https", mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] validating manifest: {
       "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
       "schemaVersion": 2,
       "config": {
          "mediaType": "application/vnd.docker.container.image.v1+json",
          "digest": "sha256:7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5b911",
          "size": 901
       },
       "layers": [
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "digest": "sha256:5720cd9c19ca69b58202945924c37c9bd7b287ce1a88882098fd59a4292e7cd9",
             "size": 296530
          }
       ]
    }
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Parsing response as Manifest: {
       "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
       "schemaVersion": 2,
       "config": {
          "mediaType": "application/vnd.docker.container.image.v1+json",
          "digest": "sha256:7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5b911",
          "size": 901
       },
       "layers": [
          {
             "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
             "digest": "sha256:5720cd9c19ca69b58202945924c37c9bd7b287ce1a88882098fd59a4292e7cd9",
             "size": 296530
          }
       ]
    }
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Pulling config layer
[2024-06-01T09:00:16Z DEBUG oci_distribution::token_cache] Fetching token registry=mcr.microsoft.com repository=oss/kubernetes/pause op=Pull miss=true
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] reuse idle connection for ("https", mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] flushed 350 bytes
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] parsed 18 headers
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body is content-length (411 bytes)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] pooling idle connection for ("https", mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG reqwest::async_impl::client] redirecting 'https://mcr.microsoft.com/v2/oss/kubernetes/pause/blobs/sha256:7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5b911' to 'https://westeurope.data.mcr.microsoft.com/42012bb2682a4d76ba7fa17a9d9a9162-qb2vm9uiex//docker/registry/v2/blobs/sha256/7b/7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5b911/data?se=2024-06-01T09%3A19%3A29Z&sig=z37JD8%2B%2F%2BSKuoc3xrmblU%2F2an1GHUkVDQKs0sI3UKrg%3D&sp=r&spr=https&sr=b&sv=2018-03-28&regid=42012bb2682a4d76ba7fa17a9d9a9162'
[2024-06-01T09:00:16Z DEBUG reqwest::connect] starting new connection: https://westeurope.data.mcr.microsoft.com/
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::dns] resolving host="westeurope.data.mcr.microsoft.com"
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::http] connecting to 204.79.197.219:443
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::http] connected to 204.79.197.219:443
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] flushed 712 bytes
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] parsed 23 headers
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body is content-length (901 bytes)
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:16Z DEBUG hyper::client::pool] pooling idle connection for ("https", westeurope.data.mcr.microsoft.com)
[2024-06-01T09:00:16Z DEBUG genpolicy::registry] digest_hash: "sha256:dcc1da6e41b612a7f7c084d7b74ec4217628481653bd312245fd482669d8a1c3"
[2024-06-01T09:00:16Z DEBUG genpolicy::registry] manifest: {
      "schemaVersion": 2,
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "digest": "sha256:7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5b911",
        "size": 901
      },
      "layers": [
        {
          "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
          "digest": "sha256:5720cd9c19ca69b58202945924c37c9bd7b287ce1a88882098fd59a4292e7cd9",
          "size": 296530
        }
      ]
    }
{
  "architecture": "amd64",
  "config": {
    "User": "65535:65535",
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Entrypoint": [
      "/pause"
    ],
    "WorkingDir": "/",
    "OnBuild": null
  },
  "created": "2021-08-31T20:59:23.003483696Z",
  "history": [
    {
      "created": "2021-08-31T20:59:23.003483696Z",
      "created_by": "ARG ARCH",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2021-08-31T20:59:23.003483696Z",
      "created_by": "ADD bin/pause-linux-amd64 /pause # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2021-08-31T20:59:23.003483696Z",
      "created_by": "USER 65535:65535",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2021-08-31T20:59:23.003483696Z",
      "created_by": "ENTRYPOINT [\"/pause\"]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:9760f55e20e3f4eb6b837e1b323b3c6f29b1ef4a4617fe98625ead879e91b1c1"
    ]
  }
}[2024-06-01T09:00:16Z INFO  genpolicy::registry] Using cache file
[2024-06-01T09:00:16Z INFO  genpolicy::registry] dm-verity root hash: 817250f1a3e336da76f5bd3fa784e1b26d959b9c131876815ba2604048b70c18
[2024-06-01T09:00:16Z DEBUG genpolicy::pod] pause container added.
[2024-06-01T09:00:16Z DEBUG genpolicy::yaml] Deployment {
        apiVersion: "apps/v1",
        kind: "Deployment",
        metadata: ObjectMeta {
            name: Some(
                "voting",
            ),
            generateName: None,
            labels: Some(
                {
                    "app.kubernetes.io/name": "voting",
                    "app.kubernetes.io/part-of": "emojivoto",
                    "app.kubernetes.io/version": "v11",
                },
            ),
            annotations: None,
            namespace: None,
        },
        spec: DeploymentSpec {
            replicas: Some(
                1,
            ),
            selector: Some(
                LabelSelector {
                    matchLabels: Some(
                        {
                            "app.kubernetes.io/name": "voting-svc",
                            "version": "v11",
                        },
                    ),
                    matchExpressions: None,
                },
            ),
            strategy: None,
            template: PodTemplateSpec {
                metadata: ObjectMeta {
                    name: None,
                    generateName: None,
                    labels: Some(
                        {
                            "app.kubernetes.io/name": "voting-svc",
                            "version": "v11",
                        },
                    ),
                    annotations: None,
                    namespace: None,
                },
                spec: PodSpec {
                    containers: [
                        Container {
                            registry: Container {
                                config_layer: DockerConfigLayer {
                                    architecture: "",
                                    config: DockerImageConfig {
                                        User: None,
                                        Tty: None,
                                        Env: None,
                                        Cmd: None,
                                        WorkingDir: None,
                                        Entrypoint: None,
                                    },
                                    rootfs: DockerRootfs {
                                        type: "",
                                        diff_ids: [],
                                    },
                                },
                                image_layers: [],
                            },
                            name: "voting-svc",
                            image: "docker.l5d.io/buoyantio/emojivoto-voting-svc:v11",
                            imagePullPolicy: None,
                            securityContext: None,
                            volumeMounts: Some(
                                [
                                    VolumeMount {
                                        mountPath: "/tls-config",
                                        name: "tls-certs",
                                        mountPropagation: None,
                                        subPathExpr: None,
                                        readOnly: None,
                                    },
                                ],
                            ),
                            env: Some(
                                [
                                    EnvVar {
                                        name: "GRPC_PORT",
                                        value: Some(
                                            "8080",
                                        ),
                                        valueFrom: None,
                                    },
                                    EnvVar {
                                        name: "PROM_PORT",
                                        value: Some(
                                            "8801",
                                        ),
                                        valueFrom: None,
                                    },
                                    EnvVar {
                                        name: "EDG_CERT_PATH",
                                        value: Some(
                                            "/tls-config/certChain.pem",
                                        ),
                                        valueFrom: None,
                                    },
                                    EnvVar {
                                        name: "EDG_CA_PATH",
                                        value: Some(
                                            "/tls-config/mesh-ca.pem",
                                        ),
                                        valueFrom: None,
                                    },
                                    EnvVar {
                                        name: "EDG_KEY_PATH",
                                        value: Some(
                                            "/tls-config/key.pem",
                                        ),
                                        valueFrom: None,
                                    },
                                ],
                            ),
                            envFrom: None,
                            resources: Some(
                                ResourceRequirements {
                                    requests: Some(
                                        {
                                            "memory": "50Mi",
                                        },
                                    ),
                                    limits: Some(
                                        {
                                            "memory": "50Mi",
                                        },
                                    ),
                                },
                            ),
                            ports: Some(
                                [
                                    ContainerPort {
                                        containerPort: 8080,
                                        hostIP: None,
                                        hostPort: None,
                                        name: Some(
                                            "grpc",
                                        ),
                                        protocol: None,
                                    },
                                    ContainerPort {
                                        containerPort: 8801,
                                        hostIP: None,
                                        hostPort: None,
                                        name: Some(
                                            "prom",
                                        ),
                                        protocol: None,
                                    },
                                ],
                            ),
                            command: None,
                            args: None,
                            lifecycle: None,
                            livenessProbe: None,
                            readinessProbe: None,
                            startupProbe: None,
                            restartPolicy: None,
                            serviceAccountName: None,
                            stdin: None,
                            tty: None,
                            terminationMessagePath: None,
                        },
                    ],
                    nodeSelector: None,
                    restartPolicy: None,
                    runtimeClassName: Some(
                        "contrast-cc-05cccd97d1eab6bf36497a6685753a04",
                    ),
                    initContainers: Some(
                        [
                            Container {
                                registry: Container {
                                    config_layer: DockerConfigLayer {
                                        architecture: "",
                                        config: DockerImageConfig {
                                            User: None,
                                            Tty: None,
                                            Env: None,
                                            Cmd: None,
                                            WorkingDir: None,
                                            Entrypoint: None,
                                        },
                                        rootfs: DockerRootfs {
                                            type: "",
                                            diff_ids: [],
                                        },
                                    },
                                    image_layers: [],
                                },
                                name: "initializer",
                                image: "ghcr.io/edgelesssys/contrast/initializer:v0.6.1@sha256:09e603f86709d36c2a89dfc38220f169197102195a9d35292a9e1fd04a92fa90",
                                imagePullPolicy: None,
                                securityContext: None,
                                volumeMounts: Some(
                                    [
                                        VolumeMount {
                                            mountPath: "/tls-config",
                                            name: "tls-certs",
                                            mountPropagation: None,
                                            subPathExpr: None,
                                            readOnly: None,
                                        },
                                    ],
                                ),
                                env: Some(
                                    [
                                        EnvVar {
                                            name: "COORDINATOR_HOST",
                                            value: Some(
                                                "coordinator",
                                            ),
                                            valueFrom: None,
                                        },
                                    ],
                                ),
                                envFrom: None,
                                resources: Some(
                                    ResourceRequirements {
                                        requests: Some(
                                            {
                                                "memory": "50Mi",
                                            },
                                        ),
                                        limits: None,
                                    },
                                ),
                                ports: None,
                                command: None,
                                args: None,
                                lifecycle: None,
                                livenessProbe: None,
                                readinessProbe: None,
                                startupProbe: None,
                                restartPolicy: None,
                                serviceAccountName: None,
                                stdin: None,
                                tty: None,
                                terminationMessagePath: None,
                            },
                            Container {
                                registry: Container {
                                    config_layer: DockerConfigLayer {
                                        architecture: "",
                                        config: DockerImageConfig {
                                            User: None,
                                            Tty: None,
                                            Env: None,
                                            Cmd: None,
                                            WorkingDir: None,
                                            Entrypoint: None,
                                        },
                                        rootfs: DockerRootfs {
                                            type: "",
                                            diff_ids: [],
                                        },
                                    },
                                    image_layers: [],
                                },
                                name: "sidecar",
                                image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy:v0.6.1@sha256:227d0be1589d3faca056d634f212d2ace7625d4682dbf33d8e6f6fcf39f16b4d",
                                imagePullPolicy: None,
                                securityContext: Some(
                                    SecurityContext {
                                        readOnlyRootFilesystem: None,
                                        allowPrivilegeEscalation: None,
                                        privileged: Some(
                                            true,
                                        ),
                                        capabilities: Some(
                                            Capabilities {
                                                add: Some(
                                                    [
                                                        "NET_ADMIN",
                                                    ],
                                                ),
                                                drop: None,
                                            },
                                        ),
                                        runAsUser: None,
                                        seccompProfile: None,
                                    },
                                ),
                                volumeMounts: Some(
                                    [
                                        VolumeMount {
                                            mountPath: "/tls-config",
                                            name: "tls-certs",
                                            mountPropagation: None,
                                            subPathExpr: None,
                                            readOnly: None,
                                        },
                                    ],
                                ),
                                env: None,
                                envFrom: None,
                                resources: None,
                                ports: None,
                                command: None,
                                args: None,
                                lifecycle: None,
                                livenessProbe: None,
                                readinessProbe: None,
                                startupProbe: None,
                                restartPolicy: Some(
                                    "Always",
                                ),
                                serviceAccountName: None,
                                stdin: None,
                                tty: None,
                                terminationMessagePath: None,
                            },
                        ],
                    ),
                    imagePullSecrets: None,
                    affinity: None,
                    volumes: Some(
                        [
                            Volume {
                                name: "tls-certs",
                                emptyDir: Some(
                                    EmptyDirVolumeSource {
                                        medium: None,
                                        sizeLimit: None,
                                    },
                                ),
                                hostPath: None,
                                persistentVolumeClaim: None,
                                configMap: None,
                                azureFile: None,
                                projected: None,
                                secret: None,
                                downwardAPI: None,
                            },
                        ],
                    ),
                    serviceAccountName: Some(
                        "voting",
                    ),
                    serviceAccount: None,
                    terminationGracePeriodSeconds: None,
                    tolerations: None,
                    hostname: None,
                    hostNetwork: None,
                    shareProcessNamespace: None,
                    dnsConfig: None,
                    dnsPolicy: None,
                    topologySpreadConstraints: None,
                    securityContext: None,
                    priorityClassName: None,
                },
            },
        },
        doc_mapping: Null,
    }
[2024-06-01T09:00:16Z INFO  genpolicy::registry] ============================================
[2024-06-01T09:00:16Z INFO  genpolicy::registry] Pulling manifest and config for "docker.l5d.io/buoyantio/emojivoto-voting-svc:v11"
[2024-06-01T09:00:16Z DEBUG genpolicy::registry] build_auth: Reference { registry: "docker.l5d.io", repository: "buoyantio/emojivoto-voting-svc", tag: Some("v11"), digest: None }
[2024-06-01T09:00:16Z DEBUG genpolicy::registry] build_auth: Docker config not found - using anonymous access.
[2024-06-01T09:00:16Z DEBUG oci_distribution::token_cache] Fetching token registry=docker.l5d.io repository=buoyantio/emojivoto-voting-svc op=Pull miss=true
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] Authorizing for image: Reference { registry: "docker.l5d.io", repository: "buoyantio/emojivoto-voting-svc", tag: Some("v11"), digest: None }
[2024-06-01T09:00:16Z DEBUG oci_distribution::client] url="https://docker.l5d.io/v2/"
[2024-06-01T09:00:16Z DEBUG reqwest::connect] starting new connection: https://docker.l5d.io/
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::dns] resolving host="docker.l5d.io"
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::http] connecting to 3.67.33.93:443
[2024-06-01T09:00:16Z DEBUG hyper::client::connect::http] connected to 3.67.33.93:443
[2024-06-01T09:00:16Z DEBUG hyper::proto::h1::io] flushed 55 bytes
[2024-06-01T09:00:17Z DEBUG hyper::proto::h1::io] parsed 7 headers
[2024-06-01T09:00:17Z DEBUG hyper::proto::h1::conn] incoming body is content-length (2 bytes)
[2024-06-01T09:00:17Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:00:17Z DEBUG hyper::client::pool] pooling idle connection for ("https", docker.l5d.io)
[2024-06-01T09:00:17Z DEBUG oci_distribution::client] Pulling image manifest from https://docker.l5d.io/v2/buoyantio/emojivoto-voting-svc/manifests/v11
[2024-06-01T09:00:17Z DEBUG oci_distribution::token_cache] Fetching token registry=docker.l5d.io repository=buoyantio/emojivoto-voting-svc op=Pull miss=true
[2024-06-01T09:00:17Z DEBUG hyper::client::pool] reuse idle connection for ("https", docker.l5d.io)
[2024-06-01T09:00:17Z DEBUG hyper::proto::h1::io] flushed 292 bytes
[2024-06-01T09:01:17Z DEBUG hyper::proto::h1::io] parsed 5 headers
[2024-06-01T09:01:17Z DEBUG hyper::proto::h1::conn] incoming body is chunked encoding
[2024-06-01T09:01:17Z DEBUG hyper::proto::h1::decode] incoming chunked header: 0x9B (155 bytes)
[2024-06-01T09:01:17Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-06-01T09:01:17Z DEBUG hyper::client::pool] pooling idle connection for ("https", docker.l5d.io)
thread 'main' panicked at src/registry.rs:115:17:
Failed to pull container image manifest and config - error: ServerError {
    code: 504,
    url: "https://docker.l5d.io/v2/buoyantio/emojivoto-voting-svc/manifests/v11",
    message: "{\n  \"errors\": [ { \"code\": \"UNKNOWN\"\n              , \"message\": \"Unable to locate the requested resource\"\n              , \"detail\": []\n              } ]\n}\n\n",
}
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Re-running the command finishes successfully (and I can find the annotations in the yaml file):

$ contrast generate deployment/
✔️ Generated workload policy annotations
✔️ Updated manifest manifest.json
katexochen commented 1 month ago

Thanks for reporting this issue. It seems like the cause for this failure lies on the container registry site, which answers with error 504: Unable to locate the requested resource. I think I've observed this error a few times myself.

The generate command (which uses the upstream genpolicy tool under the hood) will download the container manifests and layers to generate the dm-verity hashes for the policy. If the registry isn't able to serve the image, the generate command will fail.

I'm not exactly sure what causes these inconsistencies in container registries. In your case, this seems to be a problem internal to docker registry, but I've observed similar issues with ghcr.io. Sadly, there is not much we can do about it. I'll see if we can include a retry loop somewhere on our side or upstream.

Did this happen only once, or can you reproduce this issue when calling generate multiple times? Notice that the result of the generate is cached in the layers-cache.json of your workspace, so you might want to remove this file when retrying to reproduce this error.

blenessy commented 4 weeks ago

Hi @katexochen !

Did this happen only once, or can you reproduce this issue when calling generate multiple times?

Nah I cannot reproduce this, in fact I've only seen this once. Considering the 504 (gateway timeout)... I think I just got unlucky to hit a maintenance slot of docker.l5d.io.

I'll see if we can include a retry loop somewhere on our side or upstream. It is a very good idea to retry at least server errors (HTTP 5xx). I would probably also retry intermittent connection problems.

katexochen commented 4 weeks ago

From our e2e tests:

    [2024-06-05T05:45:37Z INFO  genpolicy::registry] Pulling manifest and config for "docker.l5d.io/buoyantio/emojivoto-voting-svc:v11"
                            thread 'main' panicked at src/registry.rs:116:17:
                            Failed to pull container image manifest and config - error: RequestError(
                                reqwest::Error {
                                    kind: Request,
                                    url: Url {
                                        scheme: "https",
                                        cannot_be_a_base: false,
                                        username: "",
                                        password: None,
                                        host: Some(
                                            Domain(
                                                "docker.l5d.io",
                                            ),
                                        ),
                                        port: None,
                                        path: "/v2/buoyantio/emojivoto-voting-svc/manifests/v11",
                                        query: None,
                                        fragment: None,
                                    },
                                    source: hyper::Error(
                                        Io,
                                        Os {
                                            code: 104,
                                            kind: ConnectionReset,
                                            message: "Connection reset by peer",
                                        },
                                    ),
                                },
                            )

Maybe we can upload the image somewhere else if this is specific to docker.l5d.io.