`az container create` throwing `ERROR: (RegistryErrorResponse) An error response is received from the docker registry 'index.docker.io'. Please retry later.`

[casions]

Describe the bug

When trying to create a container using the Azure CLI Container Create command, we get a RegistryErrorResponse error. This is trying to deploy an image from an Azure private registry.

This was working yesterday with the same version as today: 2.61.0

I can supply more of my own configuration on request if necessary.

Related command

az container create --resource-group $RESOURCE_GROUP --file $ACI_FILE

Errors

ERROR: (RegistryErrorResponse) An error response is received from the docker registry 'index.docker.io'. Please retry later.

Issue script & Debug output

Unable to provide this at this time.

Expected behavior

The container should be created.

Environment Summary

azure-cli 2.61.0

core 2.61.0 telemetry 1.1.0

Extensions: azure-devops 1.0.1

Dependencies: msal 1.28.0 azure-mgmt-resource 23.1.1

Python location '/opt/az/bin/python3' Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.11.8 (main, May 16 2024, 03:47:28) [GCC 11.4.0]

Additional context

Another person having the exact same issue since today with a public registry.

- task: AzureCLI@2
  name: CreateContainer
  displayName: Create container
  inputs:
    azureSubscription: ${{ parameters.subscription }}
    scriptType: bash
    scriptLocation: inlineScript
    inlineScript: |
      az container create \
        --resource-group '${{ parameters.resourceGroup }}' \
        --name '${{ parameters.name }}' \
        --image 'docker.io/mockoon/cli:latest' \
        --ports ${{ parameters.port }} \
        --location '${{ parameters.location }}' \
        --dns-name-label '${{ parameters.name }}' \
        --command-line "mockoon-cli start --data '$(publicUrl)' --port ${{ parameters.port }}"`

link

[yonzhan]

Thank you for opening this issue, we will look into it.

[danspam]

We are also experiencing this issue for at least the last 18 hours. When running the az container create command either locally in the command line or during an azure devops pipeline. The issue appears to be intermittent, failing more times that it is successful. It is not the same image all the time that fails.

Example command:

az container create --resource-group my-resource --name container-mysql-xxx --image mysql:8.0 --command-line "/usr/local/bin/docker-entrypoint.sh --lower-case-table-names=1 --collation-server=utf8mb4_unicode_ci" --environment-variables MYSQL_ROOT_PASSWORD=thepassword --location australiaeast --os-type Linux --restart-policy Never --cpu 1 --memory 1 --ports 3306 --dns-name-label container-mysql-xxx --no-wait

[Taylor-S]

Hey guys,

Thanks for looking into this. We have been experiencing this issue on and off for the past couple of weeks. But as of the past 24 hours it's been consistently failing. It's preventing our deployment process from going out. Here's what our pipeline log looks like.

Your CLI is up-to-date.
Setting AZURE_CONFIG_DIR env variable to: /home/vsts/work/_temp/.azclitask
Setting active cloud to: AzureCloud
/usr/bin/az cloud set -n AzureCloud
/usr/bin/az login --service-principal -u *** --password=*** --tenant df3ef93a-489d-4317-8cee-047a66225cc9 --allow-no-subscriptions
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "df3ef93a-489d-4317-8cee-047a66225cc9",
    "id": "df3ef93a-489d-4317-8cee-047a66225cc9",
    "isDefault": true,
    "managedByTenants": [],
    "name": "COMPANY AZURE",
    "state": "Enabled",
    "tenantId": "df3ef93a-489d-4317-8cee-047a66225cc9",
    "user": {
      "name": "***",
      "type": "servicePrincipal"
    }
  }
]
/usr/bin/az account set --subscription df3ef93a-489d-4317-8cee-047a66225cc9
/usr/bin/bash /home/vsts/work/_temp/azureclitaskscript1719962335202.sh
ERROR: (RegistryErrorResponse) An error response is received from the docker registry 'index.docker.io'. Please retry later.
Code: RegistryErrorResponse
Message: An error response is received from the docker registry 'index.docker.io'. Please retry later.
##[error]Script failed with exit code: 1
/usr/bin/az account clear
Finishing: Azure CLI 

[abarqawi]

@casions @Taylor-S @danspam This related to rate limiting Azure & Docker hub , the issue described here

https://medium.com/@alaa.barqawi/docker-rate-limit-with-azure-container-instance-and-aks-4449cede66dd

[casions]

I think I can now confirm this @abarqawi. Initially I was sceptical as we were using an image from ACR, not from the public registry. However, we have now determined that we also pull an image from the public registry at the same time and this was causing the failure due to the rate limiting as mentioned. The workaround is to copy the public image to the Azure Container Registry and pull it from there. I still don't think that we are hitting the free limit though and think that this could be caused by some sort of shared resource.

[danspam]

Thanks @abarqawi. Time to change to azure container registry...

[Taylor-S]

Great! Thanks for that @abarqawi

[avalanche-tm]

I don't understand, it says unauthenticated users can pull up to 100 images per 6-hour period. I pull 2-3 images and get this error.

[Prudhvi469]

Any update on this issue ?

[casions]

@avalanche-tm 100 pulls per 6 hours per IP address. Who else is using the IP address? https://docs.docker.com/docker-hub/download-rate-limit/