search

goharbor / harbor

An open source trusted cloud native registry project that stores, signs, and scans content.
https://goharbor.io
Apache License 2.0
23.7k stars 4.73k forks source link

[FR] proxy from registry.access.redhat.com #14401

Open Hokwang opened 3 years ago

Hokwang commented 3 years ago

Hi,

This is simple FR. Please add redhat container registry (registry.access.redhat.com) to available proxy cache registry.

Please refer to https://catalog.redhat.com/software/containers/rhel7/57ea8cee9c624c035f96f3af?container-tabs=gti&gti-tabs=unauthenticated there's an way for unauthenticated.

Many thanks,

Hokwang commented 3 years ago

@stonezdj can you consider this issue?

Today, you created docker registry type. In my test, redhat registry is not docker registry type.

bitsf commented 3 years ago

@Hokwang Do you mean you test it's not worked with docker registry type ? How did you config it and anything error ?

Hokwang commented 3 years ago

@bitsf I tested like this https://github.com/goharbor/harbor/issues/14477#issuecomment-802568582, select Quay with no ID, secret.

bitsf commented 3 years ago

I mean like this, it should work image image

Hokwang commented 3 years ago

@bitsf yeah, few days ago v2.2.2 released, I will try it.

Hokwang commented 3 years ago

@bitsf After upgrading v2.2.2,

I create registry endpoint. image

But I can't create proxy project. image

Hokwang commented 3 years ago

@bitsf something weird!!

$ k -n harbor-stage get pod harbor-stage-harbor-harbor-core-679f698c6-vh4fz -o  yaml
apiVersion: v1
kind: Pod
...
    image: goharbor/harbor-core:v2.2.2
$ k -n harbor-stage exec -it harbor-stage-harbor-harbor-core-679f698c6-vh4fz -- bash
harbor [ /harbor ]$ env | grep PERM
PERMITTED_REGISTRY_TYPES_FOR_PROXY_CACHE=docker-hub,harbor,azure-acr,aws-ecr,google-gcr,quay

I can't find docker-registry

Hokwang commented 3 years ago

@bitsf could you check this issue?

Hokwang commented 3 years ago

@stonezdj @bitsf @wy65701436 I checked v2.3.0-rc1 images, it missed docker-registry, same as v2.2.2.

$ k -n harbor-stage exec -it harbor-stage-harbor-harbor-core-8588c974bc-k8n79 -- bash
harbor [ /harbor ]$ env | grep REG
HARBOR_STAGE_HARBOR_HARBOR_REGISTRY_PORT_8001_TCP_PORT=8001
PERMITTED_REGISTRY_TYPES_FOR_PROXY_CACHE=docker-hub,harbor,azure-acr,aws-ecr,google-gcr,quay
$ k -n harbor-stage get pod harbor-stage-harbor-harbor-core-8588c974bc-k8n79 -o yaml
apiVersion: v1
kind: Pod
...
spec:
  ...
    image: goharbor/harbor-core:v2.3.0-rc1
stonezdj commented 3 years ago

Did you use the harbor-helm? It seems the env PERMITTED_REGISTRY_TYPES_FOR_PROXY_CACHE helm chart is out of date

Hokwang commented 3 years ago

@stonezdj yes, I installed harbor using helm chart.

Hokwang commented 3 years ago

OMG,

I think that is in https://github.com/goharbor/harbor/blob/448f0b6e28b00bb28758fc537cc53eb570f26af4/make/photon/prepare/templates/core/env.jinja#L42

Why helm chart's configmap has this?

Hokwang commented 3 years ago

@stonezdj @bitsf back to the original issue, I can create docker-registry type for redhat container registry.

and then I tried,

$ docker pull harbor-stage.company.net/proxy-redhat/rhel7:latest
latest: Pulling from proxy-redhat/rhel7
unsupported manifest format
$ docker pull harbor-stage.nwse.sec.samsung.net/proxy-redhat/openshift3/ose-pod:latest
latest: Pulling from proxy-redhat/openshift3/ose-pod
unsupported manifest format
$ docker pull harbor-stage.nwse.sec.samsung.net/proxy-redhat/redhat-openjdk-18/openjdk18-openshift:latest
latest: Pulling from proxy-redhat/redhat-openjdk-18/openjdk18-openshift
unsupported manifest format

it doesn't work.

Hokwang commented 3 years ago

here's core pod log,

2021-06-18T04:44:25Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 532f80d158b4e0ee9ff26f01bf37ab07 to the logger for the request GET /service/token
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token?account=luckyhk.lee&scope=repository%3Aproxy-redhat%2Frhel7%3Apull&service=harbor-registry
2021-06-18T04:44:25Z [DEBUG] [/pkg/oidc/secret.go:75]: Verifying the secret for user: user
2021-06-18T04:44:25Z [ERROR] [/server/middleware/security/oidc_cli.go:62][requestID="532f80d158b4e0ee9ff26f01bf37ab07"]: failed to verify secret, username: user, error: failed to verify the secret: secret mismatch, username: user
2021-06-18T04:44:25Z [DEBUG] [/core/auth/authenticator.go:147]: Current AUTH_MODE is oidc_auth
2021-06-18T04:44:25Z [ERROR] [/server/middleware/security/basic_auth.go:40][requestID="532f80d158b4e0ee9ff26f01bf37ab07"]: failed to authenticate user: not supported
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/security/unauthorized.go:28][requestID="532f80d158b4e0ee9ff26f01bf37ab07"]: an unauthorized security context generated for request GET /service/token
2021-06-18T04:44:25Z [DEBUG] [/core/service/token/token.go:36]: URL for token request: /service/token?account=luckyhk.lee&scope=repository%3Aproxy-redhat%2Frhel7%3Apull&service=harbor-registry
2021-06-18T04:44:25Z [DEBUG] [/core/service/token/creator.go:230]: scopes: [repository:proxy-redhat/rhel7:pull]
2021-06-18T04:44:25Z [DEBUG] [/core/service/token/authutils.go:50]: scopes: [repository:proxy-redhat/rhel7:pull]
2021/06/18 04:44:25 Model:
2021/06/18 04:44:25 r.r: sub, obj, act
2021/06/18 04:44:25 p.p: sub, obj, act, eft
2021/06/18 04:44:25 e.e: some(where (p_eft == allow)) && !some(where (p_eft == deny))
2021/06/18 04:44:25 m.m: g(r_sub, p_sub) && keyMatch2(r_obj, p_obj) && (r_act == p_act || p_act == '*')
2021/06/18 04:44:25 g.g: _, _
2021/06/18 04:44:25 Policy:
2021/06/18 04:44:25 p: sub, obj, act, eft: [[anonymous /project/4 read allow] [anonymous /project/4/label read allow] [anonymous /project/4/label list allow] [anonymous /project/4/repository list allow] [anonymous /project/4/repository pull allow] [anonymous /project/4/helm-chart read allow] [anonymous /project/4/helm-chart list allow] [anonymous /project/4/helm-chart-version read allow] [anonymous /project/4/helm-chart-version list allow] [anonymous /project/4/scan read allow] [anonymous /project/4/scanner read allow] [anonymous /project/4/tag list allow] [anonymous /project/4/artifact read allow] [anonymous /project/4/artifact list allow] [anonymous /project/4/artifact-addition read allow]]
2021/06/18 04:44:25 g: _, _: []
2021/06/18 04:44:25 Role links for: g
2021/06/18 04:44:25 
2021/06/18 04:44:25 Request: anonymous, /project/4/repository, scanner-pull ---> false
2021/06/18 04:44:25 Request: anonymous, /project/4/repository, pull ---> true
2021/06/18 04:44:25 Request: anonymous, /project/4/repository, push ---> false
2021/06/18 04:44:25 Request: anonymous, /project/4/repository, delete ---> false
2021-06-18T04:44:25Z [DEBUG] [/core/service/token/authutils.go:101]: user: , access: &{repository  proxy-redhat/rhel7 [pull]}
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id cd175fd07c227647921f8e31f3740ca1 to the logger for the request HEAD /v2/proxy-redhat/rhel7/manifests/latest
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/proxy-redhat/rhel7/manifests/latest
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 0228bce9-4a1a-46d6-8885-3a5eb03811da to the logger for the request GET /api/v2.0/ping
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/ping
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/security/unauthorized.go:28][requestID="0228bce9-4a1a-46d6-8885-3a5eb03811da"]: an unauthorized security context generated for request GET /api/v2.0/ping
2021-06-18T04:44:25Z [DEBUG] [/controller/proxy/controller.go:166]: Get the manifest list with key=cache:manifestlist:proxy-redhat/rhel7:
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 9bd0833fb4700e85b5e15af67dcdc501 to the logger for the request GET /v2/proxy-redhat/rhel7/manifests/latest
2021-06-18T04:44:25Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/proxy-redhat/rhel7/manifests/latest
2021-06-18T04:44:26Z [DEBUG] [/controller/proxy/controller.go:166]: Get the manifest list with key=cache:manifestlist:proxy-redhat/rhel7:
2021-06-18T04:44:26Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 30dcf5af75f37e51afc9fa231d3e78c4 to the logger for the request GET /v2/proxy-redhat/rhel7/manifests/sha256:d887b7391256a667bb7b7f881e82478baeb5458d240441dfc47141dbcb464d32
2021-06-18T04:44:26Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/proxy-redhat/rhel7/manifests/sha256:d887b7391256a667bb7b7f881e82478baeb5458d240441dfc47141dbcb464d32
2021-06-18T04:44:26Z [DEBUG] [/controller/proxy/controller.go:166]: Get the manifest list with key=cache:manifestlist:proxy-redhat/rhel7:
2021-06-18T04:44:30Z [DEBUG] [/server/middleware/repoproxy/proxy.go:255]: Failed to ensure tag {Repository:proxy-redhat/redhat-openjdk-18/openjdk18-openshift Reference:latest ProjectName:proxy-redhat Digest: Tag:latest BlobMountRepository: BlobMountProjectName: BlobMountDigest:} , error the artifact is not ready yet, failed to tag it to latest
2021-06-18T04:44:31Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id a045d127-1a4b-4a46-a58e-7594ea0dbc92 to the logger for the request GET /api/v2.0/ping
2021-06-18T04:44:31Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/ping
2021-06-18T04:44:31Z [DEBUG] [/server/middleware/security/unauthorized.go:28][requestID="a045d127-1a4b-4a46-a58e-7594ea0dbc92"]: an unauthorized security context generated for request GET /api/v2.0/ping
Hokwang commented 3 years ago

@stonezdj could you check upper error message?

tob123 commented 2 years ago

this issue is still there in version 2.4.0. i am also looking for a way to pull anonymously from quay using the proxy cache, but only hub.docker.com is allowed as a valid registry type.

drshawnkwang commented 2 years ago

I also have this problem with harbor v2.4.1 . I am trying to access registry.redhat.io with a username + token, (authenticated access) but cannot do so. Error message is:

docker pull harbor-foo.bar/rh-proxy/rhel7:latest
latest: Pulling from rh-proxy/rhel7
unsupported manifest format
SamCNexor commented 2 years ago

We're seeing this issue as well, on Harbor v2.4.1-c4b06d79. Can't access registry.redhat.io or registry.access.redhat.com through a proxy project.

-bash-4.2$ docker pull local-harbor.foo/registry-redhat-io/ubi7/python-38:latest
latest: Pulling from registry-redhat-io/ubi7/python-38
unsupported manifest format
-bash-4.2$ docker pull local-harbor.foo/registry-access-redhat-com/ubi7/python-38:latest
latest: Pulling from registry-access-redhat-com/ubi7/python-38
unsupported manifest format

We have several other proxy projects (e.g. hub.docker.com, quay.io) which all work fine.

github-actions[bot] commented 2 years ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

Pingoo31 commented 1 year ago

Hello, I work in a company on the deployment of an Openshift cluster. To provide the images needed for this cloud, I need to make the images from the "registry.redhat.io" registry (authenticated) available through a proxy-cache. The proxy-cache feature works with other registries. The error message is "unsupported manifest format". This point is really blocking our project and I will be disappointed to be forced to drop Harbor because of it.

tofuatjava commented 1 year ago

I found a workaround for this issue. Please, verify...

I do not know why but this works. Dirty but it works. Maybe this will help to find the route cause why podman pull local.harbor/ubi9/ubi-micro will not work in case a simple ubi9 repository was created using the endpoint

SamCNexor commented 1 year ago

I found a workaround for this issue. Please, verify...

Did not work for me, though I'm pulling using docker rather than podman which may cause some difference in behaviour.

ktzsolt commented 2 days ago

I found a workaround for this issue. Please, verify...

* create an endpoint with docker registry pointing to registry.redhat.io

* create a project called redhat using the endpoint

* pull with podman pull local.harbor/redhat/ubi9/ubi-micro

I do not know why but this works. Dirty but it works. Maybe this will help to find the route cause why podman pull local.harbor/ubi9/ubi-micro will not work in case a simple ubi9 repository was created using the endpoint

What worked for me is creating endpoint with provider Quay. image

and then creating a proxy project with (arbitrary) name "redhat", then I can pull like this: docker pull myharbor.local/redhat/ubi9/openjdk-21-runtime

When I check the repositories in the project then I can see the cached image

image

image

When first I tried with dockerhub provider it worked but the cached image was not visible in the repositories, so Quay it is!