Better non-OCI registry handling

nsmith5 commented 2 years ago

Description

We've had two folks confused because their registry was non-OCI compliant and cosign wasn't great about communicating the issue in the last week or so.

We should improve this experience by either

Returning a clear and actionable error message like "Looks like your container registry isn't OCI compliant. Try setting COSIGN_DOCKER_MEDIA_TYPES=1 and trying again" or
Detecting this automatically and just setting docker media types ourselves

elizabetht commented 2 years ago

Is there a way to detect that a given registry is non-OCI compliant?

elizabetht commented 2 years ago

Reading on @dlorenc's blog - https://dlorenc.medium.com/oci-artifacts-explained-8f4a77945c13. So absence of top-level mediaType would indicate details about a given image, but at registry level - I am still not sure how to auto-detect! Any pointers?

jdolitsky commented 2 years ago

There are a few things going here, which I'll try to clarify-

For starters, an OCI registry should return 200 OK to GET /v2/ (as described here).

Next, the top-level manifest mediaType, application/vnd.oci.image.manifest.v1+json must be accepted by the registry for manifest uploads (described here).

Then, a config.mediaType of application/vnd.oci.image.config.v1+json must be accepted by the registry.

Finally, a layer[0].mediaType of application/vnd.dev.cosign.simplesigning.v1+json must be accepted. Note that this is cosign-specific, and nothing in OCI declares that a registry must support this mediatype.

Here is an example of inspecting a cosign-produced signature manifest in GHCR:

TOKEN="$(curl -sL 'https://ghcr.io/token?scope=repository:helm/chartmuseum:pull' | jq -r .token)"

curl -sL \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Accept: application/vnd.oci.image.manifest.v1+json" \
  ghcr.io/v2/helm/chartmuseum/manifests/sha256-7ea140944c16ca05a4f4be86bd0a7e47d0163cee63ce8baafe58b921bff2e40e.sig \
  | jq .

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "size": 233,
    "digest": "sha256:b5c4db2549e44710950d3af741854ca66eecdde95812208c96ccabe01a92865c"
  },
  "layers": [
    {
      "mediaType": "application/vnd.dev.cosign.simplesigning.v1+json",
      "size": 240,
      "digest": "sha256:fbb75e70d2ff373243bafcb210ff206174077f20738fb1567b6243c1d2953d02",
      "annotations": {
        "dev.cosignproject.cosign/signature": "MEYCIQC1xoCkw6hjmqR2ZCV6H858et+egkEYpXfcOte4PAjYCAIhAKQvBdOUR/8MMGZbGtrbKIXC2ldUh9LLauJG9cijBmar",
        "dev.sigstore.cosign/bundle": "{\"SignedEntryTimestamp\":\"MEUCIDVcOsTNXXJ/1UUKa35aDpvCfR9XueOflKWzayWlY9AfAiEAqYgkFmDbCAPSB21a2j/4LEiFasHZ2lb4AF+cvsEyPag=\",\"Payload\":{\"body\":\"eyJhcGlWZXJzaW9uIjoiMC4wLjEiLCJraW5kIjoiaGFzaGVkcmVrb3JkIiwic3BlYyI6eyJkYXRhIjp7Imhhc2giOnsiYWxnb3JpdGhtIjoic2hhMjU2IiwidmFsdWUiOiJmYmI3NWU3MGQyZmYzNzMyNDNiYWZjYjIxMGZmMjA2MTc0MDc3ZjIwNzM4ZmIxNTY3YjYyNDNjMWQyOTUzZDAyIn19LCJzaWduYXR1cmUiOnsiY29udGVudCI6Ik1FWUNJUUMxeG9Da3c2aGptcVIyWkNWNkg4NThldCtlZ2tFWXBYZmNPdGU0UEFqWUNBSWhBS1F2QmRPVVIvOE1NR1piR3RyYktJWEMybGRVaDlMTGF1Skc5Y2lqQm1hciIsInB1YmxpY0tleSI6eyJjb250ZW50IjoiTFMwdExTMUNSVWRKVGlCRFJWSlVTVVpKUTBGVVJTMHRMUzB0Q2sxSlNVUkVla05EUVhCWFowRjNTVUpCWjBsVlFVMHpabUkyVlZVNWNYcEZTV1pwVEROc1V6VXJjbFJCWWs5amQwTm5XVWxMYjFwSmVtb3dSVUYzVFhjS1MycEZWazFDVFVkQk1WVkZRMmhOVFdNeWJHNWpNMUoyWTIxVmRWcEhWakpOVWtWM1JIZFpSRlpSVVVSRmQyaDZZVmRrZW1SSE9YbGFWRUZsUm5jd2VRcE5ha0Y1VFdwTmVFNTZUVEJPUkVwaFJuY3dlVTFxUVhsTmFrMTRUbnBSTUU1RVJtRk5RazE0UlZSQlVFSm5UbFpDUVc5VVEwaE9jRm96VGpCaU0wcHNDazFHYTNkRmQxbElTMjlhU1hwcU1FTkJVVmxKUzI5YVNYcHFNRVJCVVdORVVXZEJSV0ZPWTBrMllrVktUVzlGYTFOWE9FbFJTazExWjBabVkyaHpNRk1LVURWRGIwVkRlbTVGVG1NeVJXWmpMek5yTjI1V1lrVjBSbXBXWldoNE9GbGhNRUZhVUhJMmVXWnphRU5UVW1kSGVXOW9kVU5qTnpFM05rOURRV0UwZHdwblowZHhUVUUwUjBFeFZXUkVkMFZDTDNkUlJVRjNTVWhuUkVGVVFtZE9Wa2hUVlVWRVJFRkxRbWRuY2tKblJVWkNVV05FUVhwQlRVSm5UbFpJVWsxQ0NrRm1PRVZCYWtGQlRVSXdSMEV4VldSRVoxRlhRa0pVYm5veE5WRkNTRVZPVDFGMFduRmpVRXRUUzJkSlptTTFOak5xUVdaQ1owNVdTRk5OUlVkRVFWY0taMEpTV1hkQ05XWnJWVmRzV25Gc05ucEtRMmhyZVV4UlMzTllSaXRxUW1GQ1owNVdTRkpGUlZWNlFsSm9hemx2WkVoU2QyTjZiM1pNTW1Sd1pFZG9NUXBaYVRWcVlqSXdkbUZIVm5OaVV6bHFZVWRHZVdSSE1URmpNbFl4WWxNNGRWb3liREJoU0ZacFRETmtkbU50ZEcxaVJ6a3pZM2s1YVdSWGJITmFRelUxQ21KWGVFRmpiVlp0WTNrNWIxcFhSbXRqZVRsMFdWZHNkVTFDTkVkRGFYTkhRVkZSUW1jM09IZEJVVlZGUlVkb2JHSkhNSFpaTW1ob1kyNVNkR1JZVG13S1pGY3dkMGhSV1V0TGQxbENRa0ZIUkhaNlFVSkNaMUZRWTIxV2JXTjVPVzlhVjBaclkzazVkRmxYYkhWTlFrbEhRMmx6UjBGUlVVSm5OemgzUVZGSlJRcENTRUl4WXpKbmQwVjNXVXRMZDFsQ1FrRkhSSFo2UVVKQ1FWRkdXVzVXY0dKSFVYZFBVVmxMUzNkWlFrSkJSMFIyZWtGQ1FWRlJjbUZJVWpCalNFMDJDa3g1T1RCaU1uUnNZbWsxYUZrelVuQmlNalY2VEcxa2NHUkhhREZaYmxaNldsaEthbUl5TlRCYVZ6VXdURzFPZG1KVVFUSkNaMjl5UW1kRlJVRlpUeThLVFVGRlJFSkRaekJQVjBrd1RtcENhMDVYV20xYWJWVXhUMVJqTlUxNlozbFBWRlpwVGpKRk5FMTZhR3BhVjAxNlQxUkNhVTR5V1ROYVJHaHFUVUZ2UndwRFEzRkhVMDAwT1VKQlRVUkJNbWRCVFVkVlEwMVJReXRQUkRaMVRYVmhla3hFV210d2MwYzBhRFpaY1VOcU1IbFZaMHhMT1ZodlJIZHpjREJvZVRocUNqVk1OMFo1ZVV4VlduWmFibEJrUVZGdVJuRkVWMlJyUTAxQlZVSXljMVoyT1dFclNqRjZVMUV3VkVkS00wSlBPWGN3UlU1RldpdEpPSEJLZEdKcVp5OEtZa1JWZWxwcWRpOUxWbnBQZUVWQlIwVk9UMmM1YjJWVFpGRTlQUW90TFMwdExVVk9SQ0JEUlZKVVNVWkpRMEZVUlMwdExTMHRDZz09In19fX0=\",\"integratedTime\":1645637685,\"logIndex\":1490804,\"logID\":\"c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d\"}}",
        "dev.sigstore.cosign/certificate": "-----BEGIN CERTIFICATE-----\nMIIDDzCCApWgAwIBAgIUAM3fb6UU9qzEIfiL3lS5+rTAbOcwCgYIKoZIzj0EAwMw\nKjEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MREwDwYDVQQDEwhzaWdzdG9yZTAeFw0y\nMjAyMjMxNzM0NDJaFw0yMjAyMjMxNzQ0NDFaMBMxETAPBgNVBAoTCHNpZ3N0b3Jl\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEaNcI6bEJMoEkSW8IQJMugFfchs0S\nP5CoECznENc2Efc/3k7nVbEtFjVehx8Ya0AZPr6yfshCSRgGyohuCc7176OCAa4w\nggGqMA4GA1UdDwEB/wQEAwIHgDATBgNVHSUEDDAKBggrBgEFBQcDAzAMBgNVHRMB\nAf8EAjAAMB0GA1UdDgQWBBTnz15QBHENOQtZqcPKSKgIfc563jAfBgNVHSMEGDAW\ngBRYwB5fkUWlZql6zJChkyLQKsXF+jBaBgNVHREEUzBRhk9odHRwczovL2dpdGh1\nYi5jb20vaGVsbS9jaGFydG11c2V1bS8uZ2l0aHViL3dvcmtmbG93cy9idWlsZC55\nbWxAcmVmcy9oZWFkcy9tYWluMB4GCisGAQQBg78wAQUEEGhlbG0vY2hhcnRtdXNl\ndW0wHQYKKwYBBAGDvzABBgQPcmVmcy9oZWFkcy9tYWluMBIGCisGAQQBg78wAQIE\nBHB1c2gwEwYKKwYBBAGDvzABBAQFYnVpbGQwOQYKKwYBBAGDvzABAQQraHR0cHM6\nLy90b2tlbi5hY3Rpb25zLmdpdGh1YnVzZXJjb250ZW50LmNvbTA2BgorBgEEAYO/\nMAEDBCg0OWI0NjBkNWZmZmU1OTc5MzgyOTViN2E4MzhjZWMzOTBiN2Y3ZDhjMAoG\nCCqGSM49BAMDA2gAMGUCMQC+OD6uMuazLDZkpsG4h6YqCj0yUgLK9XoDwsp0hy8j\n5L7FyyLUZvZnPdAQnFqDWdkCMAUB2sVv9a+J1zSQ0TGJ3BO9w0ENEZ+I8pJtbjg/\nbDUzZjv/KVzOxEAGENOg9oeSdQ==\n-----END CERTIFICATE-----\n",
        "dev.sigstore.cosign/chain": "-----BEGIN CERTIFICATE-----\nMIIB9zCCAXygAwIBAgIUALZNAPFdxHPwjeDloDwyYChAO/4wCgYIKoZIzj0EAwMw\nKjEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MREwDwYDVQQDEwhzaWdzdG9yZTAeFw0y\nMTEwMDcxMzU2NTlaFw0zMTEwMDUxMzU2NThaMCoxFTATBgNVBAoTDHNpZ3N0b3Jl\nLmRldjERMA8GA1UEAxMIc2lnc3RvcmUwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAAT7\nXeFT4rb3PQGwS4IajtLk3/OlnpgangaBclYpsYBr5i+4ynB07ceb3LP0OIOZdxex\nX69c5iVuyJRQ+Hz05yi+UF3uBWAlHpiS5sh0+H2GHE7SXrk1EC5m1Tr19L9gg92j\nYzBhMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRY\nwB5fkUWlZql6zJChkyLQKsXF+jAfBgNVHSMEGDAWgBRYwB5fkUWlZql6zJChkyLQ\nKsXF+jAKBggqhkjOPQQDAwNpADBmAjEAj1nHeXZp+13NWBNa+EDsDP8G1WWg1tCM\nWP/WHPqpaVo0jhsweNFZgSs0eE7wYI4qAjEA2WB9ot98sIkoF3vZYdd3/VtWB5b9\nTNMea7Ix/stJ5TfcLLeABLE4BNJOsQ4vnBHJ\n-----END CERTIFICATE-----",
        "dev.sigstore.cosign/timestamp": "{\"signatures\":[{\"keyid\":\"b6710623a30c010738e64c5209d367df1c0a18cf90e6ab5292fb01680f83453d\",\"sig\":\"3046022100f666a7f4b3d85d8003f2c166e27827dfa0c4ab9282e9dab19485f4e702c61700022100dfe826e0edab5f80a40f08cc87b87777a4db30775d85684fe4950e797f2f565c\"}],\"signed\":{\"_type\":\"timestamp\",\"spec_version\":\"1.0\",\"version\":15,\"expires\":\"2022-03-08T19:14:05Z\",\"meta\":{\"snapshot.json\":{\"length\":1655,\"hashes\":{\"sha256\":\"36cf063d0717f6dc03e23027721adcd69b684d293956d3a1a7db7b0848f711d7\",\"sha512\":\"f90946d0a2dc58dae4505cfb91517a40299adf9e8719f52af187e2025aad69fcdeaeded271ec25db24869841c16fbe24f3fc56f56af8fdbb8808dccec4636b64\"},\"version\":15}}}}"
      }
    }
  ]
}

Using COSIGN_DOCKER_MEDIA_TYPES=1 only changes the top-level mediatype from application/vnd.oci.image.manifest.v1+json to application/vnd.docker.distribution.manifest.v2+json.

It doesn't necessarily mean a registry will accept the layer mediatype of application/vnd.dev.cosign.simplesigning.v1+json (although some may just match on and accept a +json suffix).

My recommendation at this point would be for cosign to create a simple test suite that can run against a given registry that produces a yes/no for support, then maintain an official list of these.

A starting point for something like this might be the distribution-spec conformance suite, potentially even extending it upstream to add custom tests. This can then be used to produce results across various registries similar to this page.

This is a complicated subject. If you are reading this, I empathize with your frustrations, and may God have mercy on your soul.

nsmith5 commented 2 years ago

This information is amazing and also low-key scary 😆 Thanks so much for the low-down info @jdolitsky ❤️

sigstore / cosign

Better non-OCI registry handling #1460