search

IBM / portieris

A Kubernetes Admission Controller for verifying image trust.
Apache License 2.0
332 stars 78 forks source link

Portieris fails to authenticate with Harbor registry notary #445

Open JerrySunWRS opened 1 year ago

JerrySunWRS commented 1 year ago

What commit ID of Portieris did you experience the problem with? 0.13.1 (f7aec93cf2c55845ce75755c9ce0b3bc08309fa0)

What went wrong? Portieris 0.13.1 was deployed and configured for a Harbor registry and notary. The image pull secret with correct credentials was accepted by Portieris, but it looks like it fails to authenticate with the notary from Harbor. Docker login and pull with the credentials stored in the secret was successful. Setting the image to public through the Harbor web interface, and allowing image pull without credentials allows Portieris to work, suggesting this is an authentication issue between Portieris and the Harbor notary. It seems like Portieris is not using the credentials in the image pull secret that was provided.

What should have happened differently? Kubernetes pod with image from Harbor registry should have been deployed successfully instead of being denied due to authorization error

How can it be reproduced? Deploy Portieris 0.13.1 with Harbor registry and notary. Try to apply a kubernetes pod with signed image from the Harbor registry.

The following imagepolicy was used

apiVersion: portieris.cloud.ibm.com/v1 kind: ImagePolicy metadata: name: allow-custom spec: repositories:

Any other relevant information There was work done in the following issue on authenticating with registries that dont support oauth: https://github.com/IBM/portieris/issues/51 The issue mentions testing against dockerhub. I am not an expert on this, but from what I've looked into, I believe the failure with the Harbor registry is because the Harbor notary authenticates differently than the dockerhub notary. The implementation in issue 51 looks like how a docker registry authenticates. Portieris seems to interact with the notary. The notary could authenticate differently from a registry. The change introduced in issue 51 seem to behave the same way in the master branch code, so the failure will likely be seen in master as well.

Portieris logs:

I0726 05:56:18.034068 1 controller.go:176] Getting policy for container image: my.harbor.registry.com:12345/my/image:my.tag namespace: default I0726 05:56:18.040977 1 controller.go:261] ImagePullSecret default/notary-harbor-secret found I0726 05:56:18.040985 1 enforcer.go:95] policy.Trust {0xc000819a37 [] https://my.harbor.registry.com:54321} I0726 05:56:18.073203 1 oauth.go:118] Calling: https://5.6.7.8:12345/service/token?scope=repository%3Amy.harbor.registry.com%3A31505%2Fmy%2Fimage%3Apull&service=harbor-notary I0726 05:56:18.121957 1 trust.go:65] GetAllTargetMetadataByName returned err: you are not authorized to perform this operation: server returned 401. I0726 05:56:18.122026 1 controller.go:176] Getting policy for container image: my.harbor.registry.com:12345/my/otherimage:my.tag namespace: default I0726 05:56:18.124795 1 controller.go:261] ImagePullSecret default/notary-harbor-secret found I0726 05:56:18.124805 1 enforcer.go:95] policy.Trust {0xc0006a1247 [] https://my.harbor.registry.com:54321} I0726 05:56:18.151565 1 oauth.go:118] Calling: https://5.6.7.8:12345/service/token?scope=repository%3Amy.harbor.registry.com%3A31505%2Fmy%2Fotherimage%3Apull&service=harbor-notary I0726 05:56:18.196796 1 trust.go:65] GetAllTargetMetadataByName returned err: you are not authorized to perform this operation: server returned 401. I0726 05:56:18.196813 1 responder.go:93] trust: policy denied the request: Deny "my.harbor.registry.com:12345/my/otherimage:my.tag", no valid ImagePullSecret defined for my.harbor.registry.com:12345/my/otherimage:my.tag I0726 05:56:18.196817 1 responder.go:93] trust: policy denied the request: Deny "my.harbor.registry.com:12345/my/image:my.tag", no valid ImagePullSecret defined for my.harbor.registry.com:12345/my/image:my.tag I0726 05:56:18.196825 1 controller.go:125] Deny for images: [my.harbor.registry.com:12345/my/image:my.tag] [my.harbor.registry.com:12345/my/otherimage:my.tag]

Notary logs:

{"go.version":"go1.14.13","http.request.host":"my.harbor.registry.com:54321","http.request.id":"805377d0-4201-4f15-b585-5e81467e047d","http.request.method":"GET","http.request.remoteaddr":"1.2.3.4","http.request.uri":"/v2/my.harbor.registry.com:12345/my/image/_trust/tuf/root.json","http.request.useragent":"portieris/1.17.12","http.response.contenttype":"application/json; charset=utf-8","http.response.duration":"267.107µs","http.response.status":401,"http.response.written":181,"level":"info","msg":"response completed","time":"2023-07-26T05:56:18Z"} {"go.version":"go1.14.13","http.request.host":"my.harbor.registry.com:54321","http.request.id":"8d0b9a3b-9bcc-4cd6-826a-e62742ef3098","http.request.method":"GET","http.request.remoteaddr":"1.2.3.4","http.request.uri":"/v2/my.harbor.registry.com:12345/my/image/_trust/tuf/root.json","http.request.useragent":"portieris/1.17.12","http.response.contenttype":"application/json; charset=utf-8","http.response.duration":"1.022095ms","http.response.status":401,"http.response.written":181,"level":"info","msg":"response completed","time":"2023-07-26T05:56:18Z"} {"go.version":"go1.14.13","http.request.host":"my.harbor.registry.com:54321","http.request.id":"e3de9368-0a8b-48ee-8eb2-7454c0007ab6","http.request.method":"GET","http.request.remoteaddr":"1.2.3.4","http.request.uri":"/v2/my.harbor.registry.com:12345/my/otherimage/_trust/tuf/root.json","http.request.useragent":"portieris/1.17.12","http.response.contenttype":"application/json; charset=utf-8","http.response.duration":"206.027µs","http.response.status":401,"http.response.written":182,"level":"info","msg":"response completed","time":"2023-07-26T05:56:18Z"} {"go.version":"go1.14.13","http.request.host":"my.harbor.registry.com:54321","http.request.id":"0ce1ad70-03f2-4d36-b369-3082e65f4338","http.request.method":"GET","http.request.remoteaddr":"1.2.3.4","http.request.uri":"/v2/my.harbor.registry.com:12345/my/otherimage/_trust/tuf/root.json","http.request.useragent":"portieris/1.17.12","http.response.contenttype":"application/json; charset=utf-8","http.response.duration":"440.249µs","http.response.status":401,"http.response.written":182,"level":"info","msg":"response completed","time":"2023-07-26T05:56:18Z"}

ARCH0911 commented 2 weeks ago

When can we expect the fix for this issue..?