Unable to access docker hub through harbor with proxy settings

richardtearle-nps commented 1 year ago

We've had harbor working for over 6 months now, as both a image repo and a proxy cache for docker hub. Recently our server was migrated to a new hosting centre, with additional security etc. after this move we're unable to access docker hub from harbor.

We use podman, podman.socket and docker-compose to deploy harbor on this server. I've configured podman to use our corp. proxy, via the /usr/share/containers/containers.conf file:

[containers]
http_proxy = true

[engine]
env = ["HTTP_PROXY=http://X.Y.160.19:8080", "HTTPS_PROXY=http://X.Y.160.19:8080"]

restarting podman service, I can reach the docker hub registry end point in a simple container, whereas without those settings I couldn't:

podman run fedora /bin/bash -c "curl --silent https://registry-1.docker.io/v2/"
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}

I then modified the harbor.yml to include our proxy settings:

# Global proxy
# Config http proxy for components, e.g. http://my.proxy.com:3128
# Components doesn't need to connect to each others via http proxy.
# Remove component from `components` array if want disable proxy
# for it. If you want use proxy for replication, MUST enable proxy
# for core and jobservice, and set `http_proxy` and `https_proxy`.
# Add domain to the `no_proxy` field, when you want disable proxy
# for some special registry.
proxy:
  http_proxy: http://X.Y.123.101:8080/
  https_proxy: http://X.Y.123.101:8080/
  no_proxy:
  components:
    - core
    - jobservice
    - trivy

Redeployed harbor, and checked the environment variables in the harbor-core container:

podman exec harbor-core env | grep -i proxy
HTTPS_PROXY=http://X.Y.123.101:8080/
NO_PROXY=trivy-adapter,localhost,postgresql,db,jobservice,notary-signer,127.0.0.1,.local,notary-server,chartmuseum,nginx,registryctl,portal,.internal,exporter,registry,log,core,redis
PERMITTED_REGISTRY_TYPES_FOR_PROXY_CACHE=docker-hub,harbor,azure-acr,aws-ecr,google-gcr,quay,docker-registry,github-ghcr,jfrog-artifactory
HTTP_PROXY=http://X.Y.123.101:8080/

and then tried the same request to reach the docker hub registry endpoint:

 podman exec harbor-core /bin/bash -c "curl --max-time 10 https://registry-1.docker.io/v2/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:10 --:--:--     0
curl: (28) Connection timed out after 10001 milliseconds

I also checked that harbor-core could reach our proxy:

podman exec harbor-core /bin/bash -c "curl --max-time 10 http://X.Y.123.101:8080/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0
curl: (28) Connection timed out after 10000 milliseconds

which it can't. Finally, just for good measure the output from a simple container:

podman run fedora /bin/bash -c "curl --max-time 10 http://10.102.123.101:8080/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3529  100  3529    0     0   538k      0 --:--:-- --:--:-- --:--:--  574k
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head>
...[html snipped]...

The output of that is as I'd have expected.

Can anyone suggest what's wrong?

TIA

zyyw commented 1 year ago

@richardtearle-nps ,

could you please also connect the proxy on the host vm where harbor is deployed: curl --max-time 10 http://X.Y.123.101:8080/ to check if the proxy is even reachable by host machine.
And alternatively, try docker run a container and do curl --max-time 10 http://X.Y.123.101:8080/ in that container.

If either 1 or 2 is connection timeout, then it should be due to your network connectivity issue.

github-actions[bot] commented 1 year ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

github-actions[bot] commented 11 months ago

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.

goharbor / harbor

Unable to access docker hub through harbor with proxy settings #18918