Error from server (BadRequest): container is waiting to start: trying and failing to pull image

[dotnetian]

When the Aspire AppHost is deployed either to a local docker-desktop cluster or to a production cluster, this error is given by pods.

PS C:\Users\Matin\Desktop\temp\AspireApp2> cd .\AspireApp2\AspireApp2.AppHost\   
PS C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost> aspirate init                      

     _                    _           ___  
    / \     ___   _ __   (_)  _ __   ( _ ) 
   / _ \   / __| | '_ \  | | | '__|  / _ \ 
  / ___ \  \__ \ | |_) | | | | |    | (_) |
 /_/   \_\ |___/ | .__/  |_| |_|     \___/ 
                 |_|                       
Handle deployments of a .NET Aspire AppHost

── Handle Initialization Defaults ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Aspirate supports Docker and Podman as container builders..
Would you like to set a fall-back value for the container builder? [y/n] (n): n
Aspirate supports setting a fall-back value for projects that have not yet set a 'ContainerRegistry' in their csproj file.
Would you like to set a fall-back value for the container registry? [y/n] (n): n
Aspirate supports setting a repository prefix for all for projects.
Would you like to set this value? [y/n] (n): n
(?) Done: Set 'Container fallback tag' to 'latest'.
Aspirate supports setting a custom directory for 'Templates' that are used when generating kustomize manifests.
Would you like to use a custom directory (selecting 'n' will default to built in templates ? [y/n] (n): n
(?) Done: Configuration for aspirate has been bootstrapped successfully at 'C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\aspirate.json'.

 ?? Execution Completed ??
PS C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost> 
PS C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost> aspirate generate                  

     _                    _           ___  
    / \     ___   _ __   (_)  _ __   ( _ )
   / _ \   / __| | '_ \  | | | '__|  / _ \
  / ___ \  \__ \ | |_) | | | | |    | (_) |
 /_/   \_\ |___/ | .__/  |_| |_|     \___/
                 |_|
Handle deployments of a .NET Aspire AppHost

── Handling Aspirate State ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

── Handling Aspirate Secrets ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Secrets are to be protected by a password

Please enter new Password: ****
Please enter it again to confirm: ****
Secret State has been initialised!.

── Handling Configuration ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Successfully loaded existing aspirate bootstrap settings from '.'.

── Handling Aspire Manifest ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Generating Aspire Manifest for supplied App Host
(?) Done:  Created Aspire Manifest At Path: C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\manifest.json

── Selecting Required Components ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

── Handling Aspire Dashboard ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Would you like to deploy the aspire dashboard and connect the OTLP endpoint? [y/n] (y): y

── Handling Inputs ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

── Handle Value and Parameter Substitution ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

── Handling DAPR Components ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
No Dapr components selected, skipping Dapr annotations.

── Gathering Information about deployables ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Gathering container details for each project in selected components
(?) Done:  Populated container details cache for project apiservice
(?) Done:  Populated container details cache for project webfrontend
Gathering Tasks Completed - Cache Populated.

── Handling Projects ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Building all project resources, and pushing containers

Executing: dotnet publish "C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\../AspireApp2.ApiService/AspireApp2.ApiService.csproj"
-p:PublishProfile="DefaultContainer" -p:PublishSingleFile="true" -p:PublishTrimmed="false" --self-contained "true" --verbosity "quiet" --nologo -r "linux-x64"
-p:ContainerRepository="apiservice" -p:ContainerImageTag="latest"
(?) Done:  Building and Pushing container for project apiservice

Executing: dotnet publish "C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\../AspireApp2.Web/AspireApp2.Web.csproj"
-p:PublishProfile="DefaultContainer" -p:PublishSingleFile="true" -p:PublishTrimmed="false" --self-contained "true" --verbosity "quiet" --nologo -r "linux-x64"
-p:ContainerRepository="webfrontend" -p:ContainerImageTag="latest"
(?) Done:  Building and Pushing container for project webfrontend
Building and push completed for all selected project components.

── Handling Dockerfiles ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
No Dockerfile components selected. Skipping build and publish action.

── Populating Secrets File ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
No secrets to protect in any selected components

── Handle Image Pull Policy ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

── Handling Namespace ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Would you like to deploy all manifests to a custom namespace? [y/n] (n): n
Generated manifests will be deployed to the default namespace.

── Handle Kustomize Manifests ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Generating kustomize manifests to run against your kubernetes cluster:
(?) Done:  Generating C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\aspirate-output\apiservice
(?) Done:  Generating C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\aspirate-output\webfrontend

── Handling Final Manifest ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Would you like to generate the top level kustomize manifest to run against your kubernetes cluster? [y/n] (y): y
Generating Aspire Dashboard manifest
(?) Done:  Generating aspirate-output/dashboard.yaml
Generating final manifest with name 'kustomization.yaml'
(?) Done:  Generating aspirate-output/kustomization.yaml

 ?? Execution Completed ??
PS C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost> aspirate apply                     

     _                    _           ___
    / \     ___   _ __   (_)  _ __   ( _ )
   / _ \   / __| | '_ \  | | | '__|  / _ \
  / ___ \  \__ \ | |_) | | | | |    | (_) |
 /_/   \_\ |___/ | .__/  |_| |_|     \___/
                 |_|
Handle deployments of a .NET Aspire AppHost

── Handling Aspirate State ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loading state from C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\aspirate-state.json.
Would you like to use all previous state values, and skip re-prompting where possible ? [y/n] (y): y
(?) Done:  State loaded successfully from C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\aspirate-state.json. Will run without re-prompting for 
values.

── Handling Aspirate Secrets ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Secrets are protected by a password. Please enter it now: ****
(?) Done:  Secret State populated successfully.

── Handle Deployment to Cluster ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Would you like to deploy the generated manifests to a kubernetes cluster defined in your kubeconfig file? [y/n] (y): y
(?) Done: Successfully set the Active Kubernetes Context to 'docker-desktop'

Executing: kubectl apply -k C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost\aspirate-output against kubernetes context docker-desktop.
configmap/apiservice-env created
configmap/webfrontend-env created
service/apiservice created
service/aspire-dashboard created
service/webfrontend created
deployment.apps/apiservice created
deployment.apps/aspire-dashboard created
deployment.apps/webfrontend created
(?) Done: Deployments successfully applied to cluster 'docker-desktop'

 ?? Execution Completed ??
PS C:\Users\Matin\Desktop\temp\AspireApp2\AspireApp2\AspireApp2.AppHost> 

And when I try to get logs of one of the services, I get this error:

kubectl --kubeconfig C:\Users\Matin\AppData\Local\JetBrains\Rider2024.1\kubernetes\2gixBXEqDZaS5505aaeeuWGBKok\config8551996372660920750.yml --context docker-desktop --namespace=default logs webfrontend-5cf9f84dd9-tsdf9 --container=webfrontend --follow=true
Error from server (BadRequest): container "webfrontend" in pod "webfrontend-5cf9f84dd9-tsdf9" is waiting to start: trying and failing to pull image

Process finished with exit code 1

[prom3theu5]

Hi Firstly, Are you running init, and saying no to everything? In that case, dont use init, its only needed if you wish to actually override some baseline values

Secondly - where are you images pushed to? Do they have a Container Registry set in the projects etc, or are they pushing directly to Docker? If its the latter, I suggest ensuring you are on the latest version of docker desktop, or at least one thats been released in the last few months (4.28+?)

Also ensure you are setting ImagePullPolicy to IfNotPresent

And if that doesn't work - its something to do with your machine. The docker shim that enables images pulling images via docker desktops kubernetes instance automatically from the docker store. Perhaps switching to the containerd backend will help but shouldn't be required.

You can always spin up a local registry, and set that in the init settings, or on command line to try

docker run -d -p 5001:5000 --restart always --name registry registry:2

then in your AppHost directory, run aspirate init

When it asks if you'd like to use a fall-back container registry, say (Y) and enter: localhost:5001