search

containrrr / watchtower

A process for automating Docker container base image updates.
https://containrrr.dev/watchtower/
Apache License 2.0
19.13k stars 852 forks source link

Panic on update #1213

Closed escoand closed 2 years ago

escoand commented 2 years ago

Describe the bug Update is failing with a panic. Not sure if this is the reason but it's complaining about a missing image I'm also not able to find with docker image inspect ...

To Reproduce

Expected behavior

Screenshots

Environment

Logs from running watchtower with the --debug option ``` time="2022-02-04T09:21:10Z" level=debug time="2022-02-04T09:21:10Z" level=debug msg="Sleeping for a second to ensure the docker api client has been properly initialized." time="2022-02-04T09:21:11Z" level=debug msg="Making sure everything is sane before starting" time="2022-02-04T09:21:11Z" level=debug msg="Retrieving running containers" time="2022-02-04T09:21:11Z" level=warning msg="Failed to retrieve container image info: Error: No such image: sha256:ec1bdc0336225ace8a5267c0bc07803d5683cddd39d173ef8323ff67e52344d6" time="2022-02-04T09:21:11Z" level=debug msg="There are no additional watchtower containers" time="2022-02-04T09:21:11Z" level=info msg="Watchtower 1.4.0" time="2022-02-04T09:21:11Z" level=info msg="Using no notifications" time="2022-02-04T09:21:11Z" level=info msg="Checking all containers (except explicitly disabled with label)" time="2022-02-04T09:21:11Z" level=info msg="Periodic runs are not enabled." time="2022-02-04T09:21:11Z" level=info msg="The HTTP API is enabled at :8080." time="2022-02-04T09:21:20Z" level=debug msg="Valid token found." time="2022-02-04T09:21:20Z" level=info msg="Updates triggered by HTTP API request." time="2022-02-04T09:21:20Z" level=debug msg="Checking containers for updated images" time="2022-02-04T09:21:20Z" level=debug msg="Retrieving running containers" time="2022-02-04T09:21:20Z" level=warning msg="Failed to retrieve container image info: Error: No such image: sha256:ec1bdc0336225ace8a5267c0bc07803d5683cddd39d173ef8323ff67e52344d6" time="2022-02-04T09:21:21Z" level=debug msg="Trying to load authentication credentials." container=/agitated_rosalind image="containrrr/watchtower:latest" time="2022-02-04T09:21:21Z" level=debug msg="No credentials for containrrr found" config_file=/config.json time="2022-02-04T09:21:21Z" level=debug msg="Got image name: containrrr/watchtower:latest" time="2022-02-04T09:21:21Z" level=debug msg="Checking if pull is needed" container=/agitated_rosalind image="containrrr/watchtower:latest" time="2022-02-04T09:21:21Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:21Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:21Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:21Z" level=debug msg="Setting scope for auth token" image=containrrr/watchtower scope="repository:containrrr/watchtower:pull" time="2022-02-04T09:21:21Z" level=debug msg="No credentials found." time="2022-02-04T09:21:22Z" level=debug msg="Parsing image ref" host=index.docker.io image=containrrr/watchtower normalized="docker.io/containrrr/watchtower:latest" tag=latest time="2022-02-04T09:21:22Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/containrrr/watchtower/manifests/latest" time="2022-02-04T09:21:22Z" level=debug msg="Found a remote digest to compare with" remote="sha256:bbf9794a691b59ed2ed3089fec53844f14ada249ee5e372ff0e595b73f4e9ab3" time="2022-02-04T09:21:22Z" level=debug msg=Comparing local="sha256:bbf9794a691b59ed2ed3089fec53844f14ada249ee5e372ff0e595b73f4e9ab3" remote="sha256:bbf9794a691b59ed2ed3089fec53844f14ada249ee5e372ff0e595b73f4e9ab3" time="2022-02-04T09:21:22Z" level=debug msg="Found a match" time="2022-02-04T09:21:22Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:22Z" level=debug msg="No new images found for /agitated_rosalind" time="2022-02-04T09:21:22Z" level=debug msg="Trying to load authentication credentials." container=/esphome image="esphome/esphome:latest" time="2022-02-04T09:21:22Z" level=debug msg="No credentials for esphome found" config_file=/config.json time="2022-02-04T09:21:22Z" level=debug msg="Got image name: esphome/esphome:latest" time="2022-02-04T09:21:22Z" level=debug msg="Checking if pull is needed" container=/esphome image="esphome/esphome:latest" time="2022-02-04T09:21:22Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:23Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:23Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:23Z" level=debug msg="Setting scope for auth token" image=esphome/esphome scope="repository:esphome/esphome:pull" time="2022-02-04T09:21:23Z" level=debug msg="No credentials found." time="2022-02-04T09:21:23Z" level=debug msg="Parsing image ref" host=index.docker.io image=esphome/esphome normalized="docker.io/esphome/esphome:latest" tag=latest time="2022-02-04T09:21:23Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/esphome/esphome/manifests/latest" time="2022-02-04T09:21:24Z" level=debug msg="Found a remote digest to compare with" remote="sha256:3df8bc7b1cbab63f9de91a17a12058ca0770d8c4ce8c908c83164a98427cb30d" time="2022-02-04T09:21:24Z" level=debug msg=Comparing local="sha256:3df8bc7b1cbab63f9de91a17a12058ca0770d8c4ce8c908c83164a98427cb30d" remote="sha256:3df8bc7b1cbab63f9de91a17a12058ca0770d8c4ce8c908c83164a98427cb30d" time="2022-02-04T09:21:24Z" level=debug msg="Found a match" time="2022-02-04T09:21:24Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:24Z" level=debug msg="No new images found for /esphome" time="2022-02-04T09:21:24Z" level=debug msg="Trying to load authentication credentials." container=/caddy image="caddy:alpine" time="2022-02-04T09:21:24Z" level=debug msg="No credentials for caddy:alpine found" config_file=/config.json time="2022-02-04T09:21:24Z" level=debug msg="Got image name: caddy:alpine" time="2022-02-04T09:21:24Z" level=debug msg="Checking if pull is needed" container=/caddy image="caddy:alpine" time="2022-02-04T09:21:24Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:24Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:24Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:24Z" level=debug msg="Setting scope for auth token" image=caddy scope="repository:library/caddy:pull" time="2022-02-04T09:21:24Z" level=debug msg="No credentials found." time="2022-02-04T09:21:24Z" level=debug msg="Parsing image ref" host=index.docker.io image=caddy normalized="docker.io/library/caddy:alpine" tag=alpine time="2022-02-04T09:21:24Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/library/caddy/manifests/alpine" time="2022-02-04T09:21:25Z" level=debug msg="Found a remote digest to compare with" remote="sha256:1659785ea84188c3ad22a6c1f96f463d167a4b81318d71f4bb6e3d37dbe73908" time="2022-02-04T09:21:25Z" level=debug msg=Comparing local="sha256:1659785ea84188c3ad22a6c1f96f463d167a4b81318d71f4bb6e3d37dbe73908" remote="sha256:1659785ea84188c3ad22a6c1f96f463d167a4b81318d71f4bb6e3d37dbe73908" time="2022-02-04T09:21:25Z" level=debug msg="Found a match" time="2022-02-04T09:21:25Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:25Z" level=debug msg="No new images found for /caddy" time="2022-02-04T09:21:25Z" level=debug msg="Trying to load authentication credentials." container=/nextcloud image="nextcloud:fpm-alpine" time="2022-02-04T09:21:25Z" level=debug msg="No credentials for nextcloud:fpm-alpine found" config_file=/config.json time="2022-02-04T09:21:25Z" level=debug msg="Got image name: nextcloud:fpm-alpine" time="2022-02-04T09:21:25Z" level=debug msg="Checking if pull is needed" container=/nextcloud image="nextcloud:fpm-alpine" time="2022-02-04T09:21:25Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:25Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:25Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:25Z" level=debug msg="Setting scope for auth token" image=nextcloud scope="repository:library/nextcloud:pull" time="2022-02-04T09:21:25Z" level=debug msg="No credentials found." time="2022-02-04T09:21:26Z" level=debug msg="Parsing image ref" host=index.docker.io image=nextcloud normalized="docker.io/library/nextcloud:fpm-alpine" tag=fpm-alpine time="2022-02-04T09:21:26Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/library/nextcloud/manifests/fpm-alpine" time="2022-02-04T09:21:26Z" level=debug msg="Found a remote digest to compare with" remote="sha256:0ff45b3f82f43f928041bc62c8fd9a4e61359a5ff63a937fbfa475f43f360657" time="2022-02-04T09:21:26Z" level=debug msg=Comparing local="sha256:b02448c82a7fec3d1d0aacbeab466707929a9acbe7c069db4dca14166878ceb1" remote="sha256:0ff45b3f82f43f928041bc62c8fd9a4e61359a5ff63a937fbfa475f43f360657" time="2022-02-04T09:21:26Z" level=debug msg="Digests did not match, doing a pull." time="2022-02-04T09:21:26Z" level=debug msg="Pulling image" container=/nextcloud image="nextcloud:fpm-alpine" time="2022-02-04T09:21:28Z" level=info msg="Found new nextcloud:fpm-alpine image (9b1db28d00c1)" time="2022-02-04T09:21:28Z" level=debug msg="Trying to load authentication credentials." container=/dyndns image="qmcgaw/ddns-updater:latest" time="2022-02-04T09:21:28Z" level=debug msg="No credentials for qmcgaw found" config_file=/config.json time="2022-02-04T09:21:28Z" level=debug msg="Got image name: qmcgaw/ddns-updater:latest" time="2022-02-04T09:21:28Z" level=debug msg="Checking if pull is needed" container=/dyndns image="qmcgaw/ddns-updater:latest" time="2022-02-04T09:21:28Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:29Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:29Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:29Z" level=debug msg="Setting scope for auth token" image=qmcgaw/ddns-updater scope="repository:qmcgaw/ddns-updater:pull" time="2022-02-04T09:21:29Z" level=debug msg="No credentials found." time="2022-02-04T09:21:29Z" level=debug msg="Parsing image ref" host=index.docker.io image=qmcgaw/ddns-updater normalized="docker.io/qmcgaw/ddns-updater:latest" tag=latest time="2022-02-04T09:21:29Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/qmcgaw/ddns-updater/manifests/latest" time="2022-02-04T09:21:29Z" level=debug msg="Found a remote digest to compare with" remote="sha256:717960e866adcf3b0fa70dd7c57c15af49af07c5e8c8f2caf71ecefddcf59e44" time="2022-02-04T09:21:29Z" level=debug msg=Comparing local="sha256:717960e866adcf3b0fa70dd7c57c15af49af07c5e8c8f2caf71ecefddcf59e44" remote="sha256:717960e866adcf3b0fa70dd7c57c15af49af07c5e8c8f2caf71ecefddcf59e44" time="2022-02-04T09:21:29Z" level=debug msg="Found a match" time="2022-02-04T09:21:29Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:29Z" level=debug msg="No new images found for /dyndns" time="2022-02-04T09:21:29Z" level=debug msg="Trying to load authentication credentials." container=/homeassistant image="docker.io/homeassistant/home-assistant:stable" time="2022-02-04T09:21:29Z" level=debug msg="No credentials for docker.io found" config_file=/config.json time="2022-02-04T09:21:29Z" level=debug msg="Got image name: docker.io/homeassistant/home-assistant:stable" time="2022-02-04T09:21:29Z" level=debug msg="Checking if pull is needed" container=/homeassistant image="docker.io/homeassistant/home-assistant:stable" time="2022-02-04T09:21:29Z" level=warning msg="Could not do a head request for \"docker.io/homeassistant/home-assistant:stable\", falling back to regular pull." container=/homeassistant image="docker.io/homeassistant/home-assistant:stable" time="2022-02-04T09:21:29Z" level=warning msg="Reason: container image info missing" container=/homeassistant image="docker.io/homeassistant/home-assistant:stable" time="2022-02-04T09:21:29Z" level=debug msg="Pulling image" container=/homeassistant image="docker.io/homeassistant/home-assistant:stable" time="2022-02-04T09:21:31Z" level=info msg="Found new docker.io/homeassistant/home-assistant:stable image (149c6cb3c7b0)" time="2022-02-04T09:21:31Z" level=info msg="Unable to update container \"/homeassistant\": no available image info. Proceeding to next." time="2022-02-04T09:21:31Z" level=debug msg="Trying to load authentication credentials." container=/ofelia image="mcuadros/ofelia:latest" time="2022-02-04T09:21:31Z" level=debug msg="No credentials for mcuadros found" config_file=/config.json time="2022-02-04T09:21:31Z" level=debug msg="Got image name: mcuadros/ofelia:latest" time="2022-02-04T09:21:31Z" level=debug msg="Checking if pull is needed" container=/ofelia image="mcuadros/ofelia:latest" time="2022-02-04T09:21:31Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:32Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:32Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:32Z" level=debug msg="Setting scope for auth token" image=mcuadros/ofelia scope="repository:mcuadros/ofelia:pull" time="2022-02-04T09:21:32Z" level=debug msg="No credentials found." time="2022-02-04T09:21:32Z" level=debug msg="Parsing image ref" host=index.docker.io image=mcuadros/ofelia normalized="docker.io/mcuadros/ofelia:latest" tag=latest time="2022-02-04T09:21:32Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/mcuadros/ofelia/manifests/latest" time="2022-02-04T09:21:33Z" level=debug msg="Found a remote digest to compare with" remote="sha256:4d67a32724f5a57393c9aca0b6731cce60b6a5b91bd369b8e41cbede2e09a3e1" time="2022-02-04T09:21:33Z" level=debug msg=Comparing local="sha256:4d67a32724f5a57393c9aca0b6731cce60b6a5b91bd369b8e41cbede2e09a3e1" remote="sha256:4d67a32724f5a57393c9aca0b6731cce60b6a5b91bd369b8e41cbede2e09a3e1" time="2022-02-04T09:21:33Z" level=debug msg="Found a match" time="2022-02-04T09:21:33Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:33Z" level=debug msg="No new images found for /ofelia" time="2022-02-04T09:21:33Z" level=debug msg="Trying to load authentication credentials." container=/mosquitto image="eclipse-mosquitto:latest" time="2022-02-04T09:21:33Z" level=debug msg="No credentials for eclipse-mosquitto:latest found" config_file=/config.json time="2022-02-04T09:21:33Z" level=debug msg="Got image name: eclipse-mosquitto:latest" time="2022-02-04T09:21:33Z" level=debug msg="Checking if pull is needed" container=/mosquitto image="eclipse-mosquitto:latest" time="2022-02-04T09:21:33Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:33Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:33Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:33Z" level=debug msg="Setting scope for auth token" image=eclipse-mosquitto scope="repository:library/eclipse-mosquitto:pull" time="2022-02-04T09:21:33Z" level=debug msg="No credentials found." time="2022-02-04T09:21:33Z" level=debug msg="Parsing image ref" host=index.docker.io image=eclipse-mosquitto normalized="docker.io/library/eclipse-mosquitto:latest" tag=latest time="2022-02-04T09:21:33Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/library/eclipse-mosquitto/manifests/latest" time="2022-02-04T09:21:34Z" level=debug msg="Found a remote digest to compare with" remote="sha256:64b7c1729f7d1fff46b5e884fc389071686b2f7ed116c3fc7e17cbcb50fa147e" time="2022-02-04T09:21:34Z" level=debug msg=Comparing local="sha256:64b7c1729f7d1fff46b5e884fc389071686b2f7ed116c3fc7e17cbcb50fa147e" remote="sha256:64b7c1729f7d1fff46b5e884fc389071686b2f7ed116c3fc7e17cbcb50fa147e" time="2022-02-04T09:21:34Z" level=debug msg="Found a match" time="2022-02-04T09:21:34Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:34Z" level=debug msg="No new images found for /mosquitto" time="2022-02-04T09:21:34Z" level=debug msg="Trying to load authentication credentials." container=/samba image="dperson/samba:latest" time="2022-02-04T09:21:34Z" level=debug msg="No credentials for dperson found" config_file=/config.json time="2022-02-04T09:21:34Z" level=debug msg="Got image name: dperson/samba:latest" time="2022-02-04T09:21:34Z" level=debug msg="Checking if pull is needed" container=/samba image="dperson/samba:latest" time="2022-02-04T09:21:34Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:34Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:34Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:34Z" level=debug msg="Setting scope for auth token" image=dperson/samba scope="repository:dperson/samba:pull" time="2022-02-04T09:21:34Z" level=debug msg="No credentials found." time="2022-02-04T09:21:34Z" level=debug msg="Parsing image ref" host=index.docker.io image=dperson/samba normalized="docker.io/dperson/samba:latest" tag=latest time="2022-02-04T09:21:34Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/dperson/samba/manifests/latest" time="2022-02-04T09:21:35Z" level=debug msg="Found a remote digest to compare with" remote="sha256:66088b78a19810dd1457a8f39340e95e663c728083efa5fe7dc0d40b2478e869" time="2022-02-04T09:21:35Z" level=debug msg=Comparing local="sha256:66088b78a19810dd1457a8f39340e95e663c728083efa5fe7dc0d40b2478e869" remote="sha256:66088b78a19810dd1457a8f39340e95e663c728083efa5fe7dc0d40b2478e869" time="2022-02-04T09:21:35Z" level=debug msg="Found a match" time="2022-02-04T09:21:35Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:35Z" level=debug msg="No new images found for /samba" time="2022-02-04T09:21:35Z" level=debug msg="Trying to load authentication credentials." container=/redis image="redis:alpine" time="2022-02-04T09:21:35Z" level=debug msg="No credentials for redis:alpine found" config_file=/config.json time="2022-02-04T09:21:35Z" level=debug msg="Got image name: redis:alpine" time="2022-02-04T09:21:35Z" level=debug msg="Checking if pull is needed" container=/redis image="redis:alpine" time="2022-02-04T09:21:35Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:36Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:36Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:36Z" level=debug msg="Setting scope for auth token" image=redis scope="repository:library/redis:pull" time="2022-02-04T09:21:36Z" level=debug msg="No credentials found." time="2022-02-04T09:21:36Z" level=debug msg="Parsing image ref" host=index.docker.io image=redis normalized="docker.io/library/redis:alpine" tag=alpine time="2022-02-04T09:21:36Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/library/redis/manifests/alpine" time="2022-02-04T09:21:36Z" level=debug msg="Found a remote digest to compare with" remote="sha256:4bed291aa5efb9f0d77b76ff7d4ab71eee410962965d052552db1fb80576431d" time="2022-02-04T09:21:36Z" level=debug msg=Comparing local="sha256:4bed291aa5efb9f0d77b76ff7d4ab71eee410962965d052552db1fb80576431d" remote="sha256:4bed291aa5efb9f0d77b76ff7d4ab71eee410962965d052552db1fb80576431d" time="2022-02-04T09:21:36Z" level=debug msg="Found a match" time="2022-02-04T09:21:36Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:36Z" level=debug msg="No new images found for /redis" time="2022-02-04T09:21:36Z" level=debug msg="Trying to load authentication credentials." container=/minidlna image="vladgh/minidlna:latest" time="2022-02-04T09:21:36Z" level=debug msg="No credentials for vladgh found" config_file=/config.json time="2022-02-04T09:21:36Z" level=debug msg="Got image name: vladgh/minidlna:latest" time="2022-02-04T09:21:36Z" level=debug msg="Checking if pull is needed" container=/minidlna image="vladgh/minidlna:latest" time="2022-02-04T09:21:36Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:37Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:37Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:37Z" level=debug msg="Setting scope for auth token" image=vladgh/minidlna scope="repository:vladgh/minidlna:pull" time="2022-02-04T09:21:37Z" level=debug msg="No credentials found." time="2022-02-04T09:21:37Z" level=debug msg="Parsing image ref" host=index.docker.io image=vladgh/minidlna normalized="docker.io/vladgh/minidlna:latest" tag=latest time="2022-02-04T09:21:37Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/vladgh/minidlna/manifests/latest" time="2022-02-04T09:21:37Z" level=debug msg="Found a remote digest to compare with" remote="sha256:1f34e4f430ab6201edd7686be4969a19655cb1e17577e543e31daffbd4366564" time="2022-02-04T09:21:37Z" level=debug msg=Comparing local="sha256:1f34e4f430ab6201edd7686be4969a19655cb1e17577e543e31daffbd4366564" remote="sha256:1f34e4f430ab6201edd7686be4969a19655cb1e17577e543e31daffbd4366564" time="2022-02-04T09:21:37Z" level=debug msg="Found a match" time="2022-02-04T09:21:37Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:37Z" level=debug msg="No new images found for /minidlna" time="2022-02-04T09:21:37Z" level=debug msg="Trying to load authentication credentials." container=/ssh_satellite image="jnovack/autossh:latest" time="2022-02-04T09:21:37Z" level=debug msg="No credentials for jnovack found" config_file=/config.json time="2022-02-04T09:21:37Z" level=debug msg="Got image name: jnovack/autossh:latest" time="2022-02-04T09:21:37Z" level=debug msg="Checking if pull is needed" container=/ssh_satellite image="jnovack/autossh:latest" time="2022-02-04T09:21:37Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:38Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:38Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:38Z" level=debug msg="Setting scope for auth token" image=jnovack/autossh scope="repository:jnovack/autossh:pull" time="2022-02-04T09:21:38Z" level=debug msg="No credentials found." time="2022-02-04T09:21:38Z" level=debug msg="Parsing image ref" host=index.docker.io image=jnovack/autossh normalized="docker.io/jnovack/autossh:latest" tag=latest time="2022-02-04T09:21:38Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/jnovack/autossh/manifests/latest" time="2022-02-04T09:21:39Z" level=debug msg="Found a remote digest to compare with" remote="sha256:f44932cde885584925364dc985769faf2ab17cf1b44271dd36dbb85ebcd1f33f" time="2022-02-04T09:21:39Z" level=debug msg=Comparing local="sha256:f44932cde885584925364dc985769faf2ab17cf1b44271dd36dbb85ebcd1f33f" remote="sha256:f44932cde885584925364dc985769faf2ab17cf1b44271dd36dbb85ebcd1f33f" time="2022-02-04T09:21:39Z" level=debug msg="Found a match" time="2022-02-04T09:21:39Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:39Z" level=debug msg="No new images found for /ssh_satellite" time="2022-02-04T09:21:39Z" level=debug msg="Trying to load authentication credentials." container=/ssh_master image="alpine:latest" time="2022-02-04T09:21:39Z" level=debug msg="No credentials for alpine:latest found" config_file=/config.json time="2022-02-04T09:21:39Z" level=debug msg="Got image name: alpine:latest" time="2022-02-04T09:21:39Z" level=debug msg="Checking if pull is needed" container=/ssh_master image="alpine:latest" time="2022-02-04T09:21:39Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:39Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:39Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:39Z" level=debug msg="Setting scope for auth token" image=alpine scope="repository:library/alpine:pull" time="2022-02-04T09:21:39Z" level=debug msg="No credentials found." time="2022-02-04T09:21:39Z" level=debug msg="Parsing image ref" host=index.docker.io image=alpine normalized="docker.io/library/alpine:latest" tag=latest time="2022-02-04T09:21:39Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/library/alpine/manifests/latest" time="2022-02-04T09:21:40Z" level=debug msg="Found a remote digest to compare with" remote="sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300" time="2022-02-04T09:21:40Z" level=debug msg=Comparing local="sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300" remote="sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300" time="2022-02-04T09:21:40Z" level=debug msg="Found a match" time="2022-02-04T09:21:40Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:40Z" level=debug msg="No new images found for /ssh_master" time="2022-02-04T09:21:40Z" level=debug msg="Trying to load authentication credentials." container=/mariadb image="yobasystems/alpine-mariadb:latest" time="2022-02-04T09:21:40Z" level=debug msg="No credentials for yobasystems found" config_file=/config.json time="2022-02-04T09:21:40Z" level=debug msg="Got image name: yobasystems/alpine-mariadb:latest" time="2022-02-04T09:21:40Z" level=debug msg="Checking if pull is needed" container=/mariadb image="yobasystems/alpine-mariadb:latest" time="2022-02-04T09:21:40Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/" time="2022-02-04T09:21:40Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized" time="2022-02-04T09:21:40Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io time="2022-02-04T09:21:40Z" level=debug msg="Setting scope for auth token" image=yobasystems/alpine-mariadb scope="repository:yobasystems/alpine-mariadb:pull" time="2022-02-04T09:21:40Z" level=debug msg="No credentials found." time="2022-02-04T09:21:41Z" level=debug msg="Parsing image ref" host=index.docker.io image=yobasystems/alpine-mariadb normalized="docker.io/yobasystems/alpine-mariadb:latest" tag=latest time="2022-02-04T09:21:41Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/yobasystems/alpine-mariadb/manifests/latest" time="2022-02-04T09:21:41Z" level=debug msg="Found a remote digest to compare with" remote="sha256:87b7e85daa15a1b8620672698e69ad50b9d4c2a2bbd9d5d9c9c4588b7b69bcd3" time="2022-02-04T09:21:41Z" level=debug msg=Comparing local="sha256:87b7e85daa15a1b8620672698e69ad50b9d4c2a2bbd9d5d9c9c4588b7b69bcd3" remote="sha256:87b7e85daa15a1b8620672698e69ad50b9d4c2a2bbd9d5d9c9c4588b7b69bcd3" time="2022-02-04T09:21:41Z" level=debug msg="Found a match" time="2022-02-04T09:21:41Z" level=debug msg="No pull needed. Skipping image." time="2022-02-04T09:21:41Z" level=debug msg="No new images found for /mariadb" 2022/02/04 09:21:41 http: panic serving 172.17.0.1:41958: runtime error: invalid memory address or nil pointer dereference goroutine 36 [running]: net/http.(*conn).serve.func1(0x10ab380) /opt/hostedtoolcache/go/1.15.15/x64/src/net/http/server.go:1801 +0xf0 panic(0x6c09f0, 0xc65968) /opt/hostedtoolcache/go/1.15.15/x64/src/runtime/panic.go:975 +0x47c github.com/containrrr/watchtower/pkg/container.Container.ImageID(...) /home/runner/work/watchtower/watchtower/pkg/container/container.go:66 github.com/containrrr/watchtower/internal/actions.stopContainersInReversedOrder(0x10e03c0, 0xe, 0x10, 0x837e80, 0x131fda0, 0x1354f38, 0x1, 0x540be400, 0x2, 0x0, ...) /home/runner/work/watchtower/watchtower/internal/actions/update.go:130 +0x184 github.com/containrrr/watchtower/internal/actions.Update(0x837e80, 0x131fda0, 0x1354f38, 0x1, 0x540be400, 0x2, 0x0, 0x0, 0x67278, 0x19350, ...) /home/runner/work/watchtower/watchtower/internal/actions/update.go:87 +0x4dc github.com/containrrr/watchtower/cmd.runUpdatesWithNotifications(0x1354f38, 0xa6b20101) /home/runner/work/watchtower/watchtower/cmd/root.go:374 +0xc0 github.com/containrrr/watchtower/cmd.Run.func1() /home/runner/work/watchtower/watchtower/cmd/root.go:192 +0x1c github.com/containrrr/watchtower/pkg/api/update.(*Handler).Handle(0x109fb90, 0x833bf8, 0x10c6aa0, 0x10b6c80) /home/runner/work/watchtower/watchtower/pkg/api/update/update.go:49 +0x178 github.com/containrrr/watchtower/pkg/api.(*API).RequireToken.func1(0x833bf8, 0x10c6aa0, 0x10b6c80) /home/runner/work/watchtower/watchtower/pkg/api/api.go:36 +0x174 net/http.HandlerFunc.ServeHTTP(0x109fba0, 0x833bf8, 0x10c6aa0, 0x10b6c80) /opt/hostedtoolcache/go/1.15.15/x64/src/net/http/server.go:2042 +0x34 net/http.(*ServeMux).ServeHTTP(0xc86978, 0x833bf8, 0x10c6aa0, 0x10b6c80) /opt/hostedtoolcache/go/1.15.15/x64/src/net/http/server.go:2417 +0x16c net/http.serverHandler.ServeHTTP(0x109ac60, 0x833bf8, 0x10c6aa0, 0x10b6c80) /opt/hostedtoolcache/go/1.15.15/x64/src/net/http/server.go:2843 +0x88 net/http.(*conn).serve(0x10ab380, 0x834ad8, 0x1097f80) /opt/hostedtoolcache/go/1.15.15/x64/src/net/http/server.go:1925 +0x7d0 created by net/http.(*Server).Serve /opt/hostedtoolcache/go/1.15.15/x64/src/net/http/server.go:2969 +0x2d0 ```

Additional context

github-actions[bot] commented 2 years ago

Hi there! 👋🏼 As you're new to this repo, we'd like to suggest that you read our code of conduct as well as our contribution guidelines. Thanks a bunch for opening your first issue! 🙏

escoand commented 2 years ago

Even with docker system prune -af it's still finding this apparently wrong image.

piksel commented 2 years ago

So, there is one container that doesn't have any image information:

21:29 DEBU Checking if pull is needed
           container=/homeassistant image="docker.io/homeassistant/home-assistant:stable"
21:29 WARN Could not do a head request for "docker.io/homeassistant/home-assistant:stable", falling back to regular pull.
21:29 WARN Reason: container image info missing
21:29 DEBU Pulling image
21:31 INFO Found new docker.io/homeassistant/home-assistant:stable image (149c6cb3c7b0)
21:31 INFO Unable to update container "/homeassistant": no available image info. Proceeding to next.

and one that it's trying to update:

21:25 DEBU Trying to load authentication credentials.
           container=/nextcloud image="nextcloud:fpm-alpine"
21:25 DEBU No credentials for nextcloud:fpm-alpine found
21:25 DEBU Got image name: nextcloud:fpm-alpine
21:25 DEBU Checking if pull is needed
21:25 DEBU Building challenge URL URL="https://index.docker.io/v2/"
21:25 DEBU Got response to challenge request
21:25 DEBU Checking challenge header content
21:25 DEBU Setting scope for auth token image=nextcloud scope="repository:library/nextcloud:pull"
21:25 DEBU No credentials found.
21:26 DEBU Parsing image ref host=index.docker.io image=nextcloud normalized="docker.io/library/nextcloud:fpm-alpine" tag=fpm-alpine
21:26 DEBU Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/library/nextcloud/manifests/fpm-alpine"
21:26 DEBU Found a remote digest to compare with
21:26 DEBU Comparing 
            local="sha256:b02448c82a7fec3d1d0aacbeab466707929a9acbe7c069db4dca14166878ceb1" 
           remote="sha256:0ff45b3f82f43f928041bc62c8fd9a4e61359a5ff63a937fbfa475f43f360657"
21:26 DEBU Digests did not match, doing a pull.
21:26 DEBU Pulling image
21:28 INFO Found new nextcloud:fpm-alpine image (9b1db28d00c1)

it then crashes while trying to stop the containers ( actions.stopContainersInReversedOrder) when trying to look up the image ID of a container. My guess is, that it's the /homeassistant container that has a dependency on another container that is being updated. We don't actually need the image info for just restarting the container though, so perhaps we could add another check to allow this to continue without crashing. That being said, you have something odd going on with your current setup. Somehow you have deleted the image that homeassistant is using, while still keeping the container running...?

escoand commented 2 years ago

I'm trying to switch from manual docker-compose up -d to a self updating setup. Maybe something went wrong while doing tests and so on. I'll stop all containers and prune everything to get a clean system again.

simskij commented 2 years ago

Closing this as we haven't heard back from OP in a couple of months.