Running kamal app exec to run migrations fails intermittently because kamal doesn't execute docker login on the remote host. In out case, the previously used ECR token expires at some point so depending on when the previous pipeline run was triggered, the next app exec may fail.
$ kamal app exec --version v1.2.3 --roles web --primary -d qa rails db:migrate
Launching command with version v1.2.3 from new container...
Running the pre-connect hook...
INFO [4b63d86e] Running /usr/bin/env .kamal/hooks/pre-connect on localhost
INFO [4b63d86e] Finished in 2.857 seconds with exit status 0 (successful).
INFO [b2bca27f] Running docker run --rm --network kamal --env SERVICE="web" ... x.dkr.ecr.eu-central-1.amazonaws.com/image:v1.2.3
ERROR (SSHKit::Command::Failed): Exception while executing on host x.x.x.x: docker exit status: 125
docker stdout: Nothing written
docker stderr: Unable to find image 'x.dkr.ecr.eu-central-1.amazonaws.com/image:v1.2.3' locally
docker: Error response from daemon: pull access denied for x.dkr.ecr.eu-central-1.amazonaws.com/image, repository does not exist or may require 'docker login': denied: Your authorization token has expired. Reauthenticate and try again.
See 'docker run --help'.
Compare it with kamal deploy:
$ kamal deploy -P --version v1.2.3 -d qa
Log into image registry...
Running the pre-connect hook...
INFO [cd18ce5e] Running /usr/bin/env .kamal/hooks/pre-connect on localhost
INFO [cd18ce5e] Finished in 3.591 seconds with exit status 0 (successful).
INFO [4631cfff] Running docker login x.dkr.ecr.eu-central-1.amazonaws.com -u [REDACTED] -p [REDACTED] on x
INFO [4631cfff] Finished in 9.694 seconds with exit status 0 (successful).
...
Kamal v2.2.2
Running
kamal app exec
to run migrations fails intermittently because kamal doesn't executedocker login
on the remote host. In out case, the previously used ECR token expires at some point so depending on when the previous pipeline run was triggered, the nextapp exec
may fail.Compare it with
kamal deploy
: