Intermittent error: Docker Login failed with exit code 1 running jobs in container

FRABUCHI commented 4 months ago

Describe the bug I'm experiencing intermittent errors when attempting to log in to Docker on GitHub Enterprise using a self-hosted runner.

The error resembles the one discussed in this issue.

I am trying to log in to Harbor, but there are no error traces in the Harbor logs corresponding to these login attempts.

When login information is incorrect, I expect an unauthorized error, but instead, I receive an error message saying "Docker login for [harbor url] failed with exit code 1".

I suspect the problem might be related to the runner. Notably, the workflow works correctly if I re-run the failed job.

To Reproduce Steps to reproduce the behavior:

Go to the job configuration.
Run the GitHub Actions workflow.
Observe the error.
Re-run the failed workflow.
Observe that it runs successfully the second time.

Expected behavior This docker login should succeed every time without exception

Runner Version and Platform

Version of your runner? 2.311.0

OS of the machine running the runner? Amazon Linux 2

What's not working?

The intermittent error occurs during the Docker login command to Harbor. The specific error is:

Docker login for [harbor url] failed with exit code 1

jobs:
  publish-events:
    runs-on:
      - gprunner
      - pool=m5a.large
    container:
      image: [harbor url]/devops/actions/devops:latest
      credentials:
        username: temp@example.com
        password: ${{ secrets.HARBOR_PASSWORD }}

The workflow works correctly when re-run.

nsudhanva commented 4 months ago

Can confirm that the same error is happening with ECR as well:

Checking docker version
Clean up resources from previous jobs
Create local container network
Starting job container
  /usr/local/bin/docker --config /runner/_work/_temp/.docker_23e[8](https://github.com/***/***/actions/runs/9921460081/job/27410175185#step:3:9)bb56-03f0-455f-ad36-3bb699f691cc login ***.dkr.ecr.us-east-1.amazonaws.com -u *** --password-stdin
  Warning: Docker login for '0[15](https://github.com/***/***/actions/runs/9921460081/job/27410175185#step:3:18)551053535.dkr.ecr.us-east-1.amazonaws.com' failed with exit code 1, back off 2.521 seconds before retry.
  /usr/local/bin/docker --config /runner/_work/_temp/.docker_23e8bb56-03f0-455f-ad36-3bb699f691cc login ***.dkr.ecr.us-east-1.amazonaws.com -u *** --password-stdin
  Warning: Docker login for '***.dkr.ecr.us-east-1.amazonaws.com' failed with exit code 1, back off 3.036 seconds before retry.
  /usr/local/bin/docker --config /runner/_work/_temp/.docker_23e8bb56-03f0-455f-ad36-3bb699f691cc login ***.dkr.ecr.us-east-1.amazonaws.com -u *** --password-stdin
  Error: Docker login for '***.dkr.ecr.us-east-1.amazonaws.com' failed with exit code 1

This I have tested and tried so far:

Refreshed credentials for ECR, updated the secrets
Made sure that the secrets and development environment are intact

Runner Version and Platform

Self hosted on k8s, Linux, latest version

spindev commented 4 months ago

We are facing the same issue with GitHub Container Registry for GHES using the normal GITHUB_TOKEN.

Runner version: 2.314.1 Runner os:

mark-pearson-bluefruit commented 4 months ago

We are facing the same issue. Here is the evidence

rtizzy commented 1 month ago

Hey folks. I wanted to chime in here as well.

We weren't facing this issue and things were working great but have started experiencing this as well.

We have a private ECR Repository that we are pulling from and we are having intermittent failures that are typically resolved by rerunning but it's a bit of a crapshoot. We are on self hosted runners utilizing RunsOn (AWS)

Starting job container
  /usr/bin/docker --config /home/runner/_work/_temp/.docker_75a837a8-3062-424d-b10b-595fc1b2770e login 934464144164.dkr.ecr.us-east-1.amazonaws.com -u AWS --password-stdin
  Warning: Docker login for '934464144[16](https://github.com/go-highway/highway-monitor/actions/runs/11410818716/job/31816166994#step:3:19)4.dkr.ecr.us-east-1.amazonaws.com' failed with exit code 1, back off 7.076 seconds before retry.
  /usr/bin/docker --config /home/runner/_work/_temp/.docker_75a837a8-3062-424d-b10b-595fc1b2770e login 934464144164.dkr.ecr.us-east-1.amazonaws.com -u AWS --password-stdin
  Warning: Docker login for '934464144164.dkr.ecr.us-east-1.amazonaws.com' failed with exit code 1, back off 8.022 seconds before retry.
  /usr/bin/docker --config /home/runner/_work/_temp/.docker_75a837a8-3062-424d-b10b-595fc1b2770e login 934464144164.dkr.ecr.us-east-1.amazonaws.com -u AWS --password-stdin
  Error: Docker login for '934464144164.dkr.ecr.us-east-1.amazonaws.com' failed with exit code 1

actions / runner