search

metal3-io / metal3-dev-env

Metal³ Development Environment
Apache License 2.0
113 stars 118 forks source link

Default to quay.io for image hub and registry image #1473

Open elfosardo opened 4 days ago

elfosardo commented 4 days ago

This should help avoid issues related to dockerhub rate limits.

metal3-io-bot commented 4 days ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please ask for approval from elfosardo. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/metal3-io/metal3-dev-env/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
elfosardo commented 4 days ago

/cc @dtantsur @Rozzii

elfosardo commented 4 days ago

a partial test has been successfully done in ironic CI https://review.opendev.org/c/openstack/ironic/+/935895 there we switched to quay.io as docker.io was too often having issues due to rate limits

elfosardo commented 4 days ago

/test metal3-centos-e2e-integration-test-release-1-8 metal3-dev-env-integration-test-ubuntu-main

tuminoid commented 4 days ago

DOCKER_HUB_PROXY is configured also in project-infra and CAPM3. One is pointing to docker.io the other is pointing via our Nordix registry proxy. So not 100% sure how this actually changes where we pull stuff during tests.

tuminoid commented 4 days ago

/cc @Sunnatillo PTAL

metal3-io-bot commented 4 days ago

@elfosardo: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
metal3-centos-e2e-integration-test-release-1-8 14debc32d71b4960a17a03495151742a1a12c716 link true /test metal3-centos-e2e-integration-test-release-1-8
metal3-dev-env-integration-test-ubuntu-main 14debc32d71b4960a17a03495151742a1a12c716 link true /test metal3-dev-env-integration-test-ubuntu-main
Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
metal3-io-bot commented 4 days ago

@elfosardo: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
metal3-centos-e2e-integration-test-release-1-8 14debc32d71b4960a17a03495151742a1a12c716 link true /test metal3-centos-e2e-integration-test-release-1-8
metal3-dev-env-integration-test-ubuntu-main 14debc32d71b4960a17a03495151742a1a12c716 link true /test metal3-dev-env-integration-test-ubuntu-main

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
tuminoid commented 4 days ago

This is also not working as-is:

[2024-11-22T08:42:23.067Z] ++ sudo docker pull quay.io/kindest/node:v1.31.0
[2024-11-22T08:42:28.015Z] Error response from daemon: unauthorized: access to the requested resource is not authorized
lentzi90 commented 4 days ago

Yes we need to sync this with https://github.com/metal3-io/project-infra/blob/00a638ebab1a7dff1a87fcb6a6e7c5871425271d/jenkins/scripts/integration_test_env.sh#L65-L67 I was under the impression that we already used the Nordix proxies in all dev-env tests. Where are we seeing issues? In BMO e2e I know it is not configured yet, but it hasn't been an issue there yet as far as I know

elfosardo commented 4 days ago

@lentzi90 @tuminoid we could assign the value of DOCKER_HUB_PROXY in project-infra to CONTAINER_HUB_PROXY and then change the logic once this merges something like export CONTAINER_HUB_PROXY=$DOCKER_HUB_PROXY right after https://github.com/metal3-io/project-infra/blob/00a638ebab1a7dff1a87fcb6a6e7c5871425271d/jenkins/scripts/integration_test_env.sh#L65-L67

elfosardo commented 4 days ago

This is also not working as-is:

[2024-11-22T08:42:23.067Z] ++ sudo docker pull quay.io/kindest/node:v1.31.0
[2024-11-22T08:42:28.015Z] Error response from daemon: unauthorized: access to the requested resource is not authorized

@tuminoid right so that is not migrated yet to non rate limited repo, they are actually taking advantage of the docker open source project participation to get unlimited rate, so we could maybe hardcode docker.io there

tuminoid commented 4 days ago

This is also not working as-is:

[2024-11-22T08:42:23.067Z] ++ sudo docker pull quay.io/kindest/node:v1.31.0
[2024-11-22T08:42:28.015Z] Error response from daemon: unauthorized: access to the requested resource is not authorized

@tuminoid right so that is not migrated yet to non rate limited repo, they are actually taking advantage of the docker open source project participation to get unlimited rate, so we could maybe hardcode docker.io there

Not a fan of hardcoding as we try to utilize the proxy to not stress any registry. (We do have a lot of work to do for that, I give you that.)

elfosardo commented 4 days ago

This is also not working as-is:

[2024-11-22T08:42:23.067Z] ++ sudo docker pull quay.io/kindest/node:v1.31.0
[2024-11-22T08:42:28.015Z] Error response from daemon: unauthorized: access to the requested resource is not authorized

@tuminoid right so that is not migrated yet to non rate limited repo, they are actually taking advantage of the docker open source project participation to get unlimited rate, so we could maybe hardcode docker.io there

Not a fan of hardcoding as we try to utilize the proxy to not stress any registry. (We do have a lot of work to do for that, I give you that.)

not a fan of hardcoded things either, but I don't see other solution if not dropping this, and I strongly believe that docker rate limits is way worse in terms of "stress" :)

tuminoid commented 4 days ago

This is also not working as-is:

[2024-11-22T08:42:23.067Z] ++ sudo docker pull quay.io/kindest/node:v1.31.0
[2024-11-22T08:42:28.015Z] Error response from daemon: unauthorized: access to the requested resource is not authorized

@tuminoid right so that is not migrated yet to non rate limited repo, they are actually taking advantage of the docker open source project participation to get unlimited rate, so we could maybe hardcode docker.io there

Not a fan of hardcoding as we try to utilize the proxy to not stress any registry. (We do have a lot of work to do for that, I give you that.)

not a fan of hardcoded things either, but I don't see other solution if not dropping this, and I strongly believe that docker rate limits is way worse in terms of "stress" :)

Point being, Is there is a reason not use the proxy env instead, so it would go via Harbor? Can't dig into the code right now myself.

elfosardo commented 4 days ago

@tuminoid ok, I think I skipped a step here, yes, we can use the proxy and we should wouldn't that mean keeping DOCKER_HUB_PROXY variable too? the kindest/node image needs docker.io

tuminoid commented 1 day ago

@tuminoid ok, I think I skipped a step here, yes, we can use the proxy and we should wouldn't that mean keeping DOCKER_HUB_PROXY variable too? the kindest/node image needs docker.io

Yes. With proxy used in more places than it is now, technically we wouldn't need to move to quay.io either. We're depending on both anyways, so it doesn't matter which one we use more. We should just use proxy more than we are now.