search

containrrr / watchtower

A process for automating Docker container base image updates.
https://containrrr.dev/watchtower/
Apache License 2.0
18.56k stars 834 forks source link

Error Could not do a head request for #1528

Closed stanthewizzard closed 1 year ago

stanthewizzard commented 1 year ago

NOTE: See https://github.com/containrrr/watchtower/discussions/1529 for workarounds until this gets properly fixed

Describe the bug

Hello, since 48h hour, when auto updating I get this for every container that need an update:

Could not do a head request for "xxxx", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

thanks for help

Steps to reproduce

docker run -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once

Expected behavior

update with correct message pull

Note: The image does get updated, you only get a warning that it couldn't do so without using up the docker API rate limit -@piksel

Environment

Your logs

Logs ```text 2023-01-20T21:19:57Z [I] Watchtower 1.5.1 2023-01-20T21:19:57Z [I] Using no notifications 2023-01-20T21:19:57Z [I] Checking all containers (except explicitly disabled with label) 2023-01-20T21:19:57Z [I] Running a one time update. 2023-01-20T21:19:59Z [W] Could not do a head request for "pihole/pihole:latest", falling back to regular pull. container: /pihole image: pihole/pihole:latest 2023-01-20T21:19:59Z [W] Reason: registry responded to head request with "404 Not Found", auth: "not present" container: /pihole image: pihole/pihole:latest 2023-01-20T21:20:07Z [I] Session done Failed: 0 Scanned: 13 Updated: 0 notify: no 2023-01-20T21:20:07Z [I] Waiting for the notification goroutine to finish notify: no ```
github-actions[bot] commented 1 year ago

Hi there! 👋🏼 As you're new to this repo, we'd like to suggest that you read our code of conduct as well as our contribution guidelines. Thanks a bunch for opening your first issue! 🙏

darthShadow commented 1 year ago

This may be due to the recent buildx change due to the Github Ubuntu Runner being upgraded:

https://github.com/docker/build-push-action/issues/755

This results in images being pushed whose manifests cannot be parsed due to these issues:

https://github.com/moby/moby/issues/43126

https://github.com/docker/buildx/issues/1509

I didn't get time to verify via the API for a failing image (https://github.com/AnalogJ/scrutiny, specifically the collector) but the docker manifest inspect command does fail for it and docker buildx imagetools inspect shows that it's an oci image.

darthShadow commented 1 year ago

I have verified these images of the reports from the linked thread and all have the above issue:

koenkk/zigbee2mqtt:latest netdata/netdata:latest pihole/pihole:latest crowdsecurity/crowdsec:latest

stanthewizzard commented 1 year ago

Yes I have it too with pihole. So nothing to do on my side (edit:removed the ?) thank you

darthShadow commented 1 year ago

They (pihole) can pin the buildx version to v0.9.1 to fix their images (Example: https://github.com/cilium/cilium/pull/23206)

stanthewizzard commented 1 year ago

As I have some containers exhibiting this behavior it will not be easy. I posted on pihole git

dreary-ennui commented 1 year ago

Confirmed, seeing it on crowdsecurity/crowdsec:latest

MilesTEG1 commented 1 year ago

Hello 👋🏻 I too have those messages since several days. The image impacted varies from time to to time.

Here the email notification I received this morning : ```log Could not do a head request for "ttionya/vaultwarden-backup:latest", falling back to regular pull. Reason: registry responded to head request with "404 Not Found", auth: "not present" Could not do a head request for "crowdsecurity/crowdsec:latest", falling back to regular pull. Reason: registry responded to head request with "404 Not Found", auth: "not present" Could not do a head request for "[ghcr.io/lissy93/dashy:latest](https://github.com/containrrr/watchtower/issues/ghcr.io/lissy93/dashy:latest)", falling back to regular pull. Reason: registry responded to head request with "404 Not Found", auth: "not present" Found new [ghcr.io/lissy93/dashy:latest](https://github.com/containrrr/watchtower/issues/ghcr.io/lissy93/dashy:latest) image (1edbe1a164c7) Stopping /dashy (1f1e9f26a75b) with SIGTERM Creating /dashy Removing image 0d5a71213f81 ```
And the one form yesterday: ```log Found new gitea/gitea:latest image (86b1df821fa3) Could not do a head request for "crowdsecurity/crowdsec:latest", falling back to regular pull. Reason: registry responded to head request with "404 Not Found", auth: "not present" Found new [lscr.io/linuxserver/swag:latest](https://github.com/containrrr/watchtower/issues/lscr.io/linuxserver/swag:latest) image (7f3a0574ee4e) Could not do a head request for "ttionya/vaultwarden-backup:latest", falling back to regular pull. Reason: registry responded to head request with "404 Not Found", auth: "not present" Found new ttionya/vaultwarden-backup:latest image (5762f8c64fd9) Found new adguard/adguardhome:latest image (6d0497b485db) Stopping /adguardhome_macvlan (d8260a3b81e7) with SIGTERM Stopping /vaultwarden_backup_ttionya (f93b66cbed33) with SIGTERM Stopping /swag (920c755314b8) with SIGTERM Stopping /gitea (eb454549b72f) with SIGTERM Creating /gitea Creating /swag Creating /vaultwarden_backup_ttionya Creating /adguardhome_macvlan Removing image 3591480978a0 Removing image d9466ff4a8c3 Removing image e67bbb3e18ae Removing image f0fd1c319729 ```

I understand that the update are still done, right ?

And there is nothing we can do to not have those errors , correct too ? Because if we have to fill issue for all images that are generating this head request error, it will be a pain in the…

darthShadow commented 1 year ago

Yeah, from my understanding, either the images need to fix it or we need to wait for the above linked issues to be resolved by docker.

In the meanwhile, you could hide these warnings by setting WATCHTOWER_WARN_ON_HEAD_FAILURE to "never" (https://containrrr.dev/watchtower/arguments/#head_failure_warnings)

stanthewizzard commented 1 year ago

I added in my docker compose WATCHTOWER_WARN_ON_HEAD_FAILURE: "never"

then docker-compose up -d

and I still get this: ``` 2023-01-21T09:37:08Z [W] Could not do a head request for "pihole/pihole:latest", falling back to regular pull. container: /pihole image: pihole/pihole:latest 2023-01-21T09:37:08Z [W] Reason: registry responded to head request with "404 Not Found", auth: "not present" container: /pihole image: pihole/pihole:latest ```
Unrepentant-Atheist commented 1 year ago

I have it too. It started around November 2022 doing this.

crazy-max commented 1 year ago

They (pihole) can pin the buildx version to v0.9.1 to fix their images (Example: cilium/cilium#23206)

No need to pin to previous release v0.9.1, pihole repo can just disable provenance as shown in https://github.com/docker/buildx/issues/1513#issuecomment-1398340039. Also OCI images are around for a few years now so I think Watchtower should support them instead.

stanthewizzard commented 1 year ago

@crazy-max They don't want to do anything apparently because this is not a pi-hole bug.

stanthewizzard commented 1 year ago

The pi-hole issue is no more Thanks to @PromoFaux

piksel commented 1 year ago

Also OCI images are around for a few years now so I think Watchtower should support them instead.

Yeah, but we are using the docker client's own code, so as long as docker manifest inspect doesn't support it, we probably won't either. 

> docker manifest inspect -v crowdsecurity/crowdsec:latest
no such manifest: docker.io/crowdsecurity/crowdsec:latest
piksel commented 1 year ago

This actually looks to be quite an easy fix, since only adding the application/vnd.oci.image.index.v1+json mime to type the Accepted: header seems to make it work correctly.

We don't supporting parsing the response, but that is irrelevant, since the only thing we are interested in is if it's been changed since the image was pulled.

crazy-max commented 1 year ago

Also OCI images are around for a few years now so I think Watchtower should support them instead.

Yeah, but we are using the docker client's own code, so as long as docker manifest inspect doesn't support it, we probably won't either.

> docker manifest inspect -v crowdsecurity/crowdsec:latest
no such manifest: docker.io/crowdsecurity/crowdsec:latest

Yes this issue is being tracked in https://github.com/moby/moby/issues/43126 (cc @thaJeztah)

vilmotz commented 1 year ago

With the homarr image is the same :( https://github.com/ajnart/homarr

tamimology commented 1 year ago

I have verified these images of the reports from the linked thread and all have the above issue:

koenkk/zigbee2mqtt:latest netdata/netdata:latest ~pihole/pihole:latest~ crowdsecurity/crowdsec:latest

I would like to add the following to the list

amir20/dozzle:latest jokobsk/pi.alert:latest

evilscientress commented 1 year ago

esphome/esphome:latest is also affected by the issue.

There also is an explicit warning for this issue in the release notes of buildx 0.10.0. Seams like many have updated without checking them or just use the latest version in their CI. I do the later myself. Would not expect that a breaking change like that is released until at least dockers own registry is ready for it.

MilesTEG1 commented 1 year ago

Hello, here some images that cause the head request with "404 Not Found", auth: "not present" (some have already been reported)

mytelegrambot commented 1 year ago

here some images that cause the head request with "404 Not Found", auth: "not present" https://github.com/wiserain/docker-flexget

tweek11 commented 1 year ago

Was this resolved or expected to be with a upcoming update? Sorry for my confusion - but I've been reading through the post and others where it has been mentioned - But still confused. Again, my apology.

I ask because I've been getting the following for a little while now when manually running Watchtower. This was just dispayed before posting here.

time="2023-01-28T21:56:30Z" level=warning msg="Could not do a head request for \"netboxcommunity/netbox:v3.4-2.4.0\", falling back to regular pull." container=/netbox-docker_netbox-housekeeping_1 image="netboxcommunity/netbox:v3.4-2.4.0" time="2023-01-28T21:56:30Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/netbox-docker_netbox-housekeeping_1 image="netboxcommunity/netbox:v3.4-2.4.0" time="2023-01-28T21:56:32Z" level=warning msg="Could not do a head request for \"netboxcommunity/netbox:v3.4-2.4.0\", falling back to regular pull." container=/netbox-docker_netbox-worker_1 image="netboxcommunity/netbox:v3.4-2.4.0" time="2023-01-28T21:56:32Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/netbox-docker_netbox-worker_1 image="netboxcommunity/netbox:v3.4-2.4.0" time="2023-01-28T21:56:34Z" level=warning msg="Could not do a head request for \"netboxcommunity/netbox:v3.4-2.4.0\", falling back to regular pull." container=/netbox-docker_netbox_1 image="netboxcommunity/netbox:v3.4-2.4.0" time="2023-01-28T21:56:34Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/netbox-docker_netbox_1 image="netboxcommunity/netbox:v3.4-2.4.0" time="2023-01-28T21:56:38Z" level=warning msg="Could not do a head request for \"amir20/dozzle:latest\", falling back to regular pull." container=/dozzle image="amir20/dozzle:latest" time="2023-01-28T21:56:38Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/dozzle image="amir20/dozzle:latest" time="2023-01-28T21:56:42Z" level=warning msg="Could not do a head request for \"ghcr.io/paperless-ngx/paperless-ngx:latest\", falling back to regular pull." container=/paperless_webserver_1 image="ghcr.io/paperless-ngx/paperless-ngx:latest" time="2023-01-28T21:56:42Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/paperless_webserver_1 image="ghcr.io/paperless-ngx/paperless-ngx:latest"

tessierp commented 1 year ago

I'm experiencing the exact same issue.

tamimology commented 1 year ago

I'm experiencing the exact same issue.

Add this in the env in your docker-compose for a quick and temporary fix

environment:
  WATCHTOWER_WARN_ON_HEAD_FAILURE: "never"
tessierp commented 1 year ago

I'm experiencing the exact same issue.

Add this in the env in your docker-compose for a quick and temporary fix

environment:
  WATCHTOWER_WARN_ON_HEAD_FAILURE: "never"

Unfortunately that doesn't solve the issue.

piksel commented 1 year ago

@tessierp, I don't think you have the same issue, since this was patched (hence why this is closed). If you create a question in discussions with details on your environment, logs and error message we might be able to help you better.

tessierp commented 1 year ago

Hi @piksel , here is my configuration 👍🏻 

version: '3.8'
services:
  watchtower:
    deploy:
      placement:
        constraints:
          - node.role==manager
      restart_policy:
        condition: any
        max_attempts: 100
        delay: 30s
    image: containrrr/watchtower
    environment:
      - WATCHTOWER_WARN_ON_HEAD_FAILURE="never"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - sysmaint-net

networks:
  sysmaint-net:
    driver: overlay
    attachable: true

time="2023-04-10T12:10:47Z" level=info msg="Watchtower 1.5.3"
time="2023-04-10T12:10:47Z" level=info msg="Using no notifications"
time="2023-04-10T12:10:47Z" level=info msg="Checking all containers (except explicitly disabled with label)"
time="2023-04-10T12:10:47Z" level=info msg="Scheduling first run: 2023-04-10 12:11:17 +0000 UTC"
time="2023-04-10T12:10:47Z" level=info msg="Note that the first check will be performed in 29 seconds"
time="2023-04-10T12:11:17Z" level=warning msg="Could not do a head request for \"containrrr/watchtower:latest@sha256:0ca7a88fd0748aa6f32e50b67eb11148cdb989fc595264c2778c85297a2c1abe\", falling back to regular pull." container=/sysmaint_watchtower.1.4sko7jbkhkapl2ttgmigrfw1j image="containrrr/watchtower:latest@sha256:0ca7a88fd0748aa6f32e50b67eb11148cdb989fc595264c2778c85297a2c1abe"
time="2023-04-10T12:11:17Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmaint_watchtower.1.4sko7jbkhkapl2ttgmigrfw1j image="containrrr/watchtower:latest@sha256:0ca7a88fd0748aa6f32e50b67eb11148cdb989fc595264c2778c85297a2c1abe"
time="2023-04-10T12:11:17Z" level=warning msg="Could not do a head request for \"mietzen/bastillion:3.14.0@sha256:51a2641dac3d5fbbc4570ff48641437b54fa48ae6cf2f122439960f6032b8c21\", falling back to regular pull." container=/sysmaint_bastillion.1.74zmrju0y46f1zg5fve8ah0ko image="mietzen/bastillion:3.14.0@sha256:51a2641dac3d5fbbc4570ff48641437b54fa48ae6cf2f122439960f6032b8c21"
time="2023-04-10T12:11:17Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmaint_bastillion.1.74zmrju0y46f1zg5fve8ah0ko image="mietzen/bastillion:3.14.0@sha256:51a2641dac3d5fbbc4570ff48641437b54fa48ae6cf2f122439960f6032b8c21"
time="2023-04-10T12:11:17Z" level=warning msg="Could not do a head request for \"portainer/agent:latest@sha256:f763a45e6033b67b142d8dd8c5a4c450a514df2ea2bc6bffce8d671abd2341b8\", falling back to regular pull." container=/portainer_agent.s7av0vo9mflevxvx6jfk70tb0.mj2ldpaux5om1u4nbznir6mv9 image="portainer/agent:latest@sha256:f763a45e6033b67b142d8dd8c5a4c450a514df2ea2bc6bffce8d671abd2341b8"
time="2023-04-10T12:11:17Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/portainer_agent.s7av0vo9mflevxvx6jfk70tb0.mj2ldpaux5om1u4nbznir6mv9 image="portainer/agent:latest@sha256:f763a45e6033b67b142d8dd8c5a4c450a514df2ea2bc6bffce8d671abd2341b8"
time="2023-04-10T12:11:18Z" level=warning msg="Could not do a head request for \"portainer/portainer-ce:latest@sha256:9fa1ec78b4e29d83593cf9720674b72829c9cdc0db7083a962bc30e64e27f64e\", falling back to regular pull." container=/portainer_portainer.1.5mzsv9cqxn9ohcu0lcqqdebra image="portainer/portainer-ce:latest@sha256:9fa1ec78b4e29d83593cf9720674b72829c9cdc0db7083a962bc30e64e27f64e"
time="2023-04-10T12:11:18Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/portainer_portainer.1.5mzsv9cqxn9ohcu0lcqqdebra image="portainer/portainer-ce:latest@sha256:9fa1ec78b4e29d83593cf9720674b72829c9cdc0db7083a962bc30e64e27f64e"
time="2023-04-10T12:11:18Z" level=warning msg="Could not do a head request for \"zabbix/zabbix-web-nginx-pgsql:latest@sha256:3698815efa78501143bb9be06c4c06289242ca6fcfb682c1ebdb1325fb2a4441\", falling back to regular pull." container=/sysmon_zabbix-web.1.d0ogoi6p39n4uva8s3tls9nld image="zabbix/zabbix-web-nginx-pgsql:latest@sha256:3698815efa78501143bb9be06c4c06289242ca6fcfb682c1ebdb1325fb2a4441"
time="2023-04-10T12:11:18Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmon_zabbix-web.1.d0ogoi6p39n4uva8s3tls9nld image="zabbix/zabbix-web-nginx-pgsql:latest@sha256:3698815efa78501143bb9be06c4c06289242ca6fcfb682c1ebdb1325fb2a4441"
time="2023-04-10T12:11:18Z" level=warning msg="Could not do a head request for \"orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8\", falling back to regular pull." container=/tools_intranetcms.1.l8z3qv0axj0lrn4b1ottubzmh image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:18Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_intranetcms.1.l8z3qv0axj0lrn4b1ottubzmh image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:18Z" level=warning msg="Could not do a head request for \"teamspeak:latest@sha256:43ae19ea49a5af4870beb62f249d9dfa64a9a4e5d66a6bf421b7986cbd03d142\", falling back to regular pull." container=/comms_teamspeak.1.q408s3o7eucltpwqm1yektzy7 image="teamspeak:latest@sha256:43ae19ea49a5af4870beb62f249d9dfa64a9a4e5d66a6bf421b7986cbd03d142"
time="2023-04-10T12:11:18Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_teamspeak.1.q408s3o7eucltpwqm1yektzy7 image="teamspeak:latest@sha256:43ae19ea49a5af4870beb62f249d9dfa64a9a4e5d66a6bf421b7986cbd03d142"
time="2023-04-10T12:11:19Z" level=warning msg="Could not do a head request for \"thelounge/thelounge:latest@sha256:16d6203694c029db77a5ddb2d81ddc741977329268de17fe8b51b1363e7ee952\", falling back to regular pull." container=/comms_thelounge.1.zlol0hnjro2fis191td950isb image="thelounge/thelounge:latest@sha256:16d6203694c029db77a5ddb2d81ddc741977329268de17fe8b51b1363e7ee952"
time="2023-04-10T12:11:19Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_thelounge.1.zlol0hnjro2fis191td950isb image="thelounge/thelounge:latest@sha256:16d6203694c029db77a5ddb2d81ddc741977329268de17fe8b51b1363e7ee952"
time="2023-04-10T12:11:19Z" level=warning msg="Could not do a head request for \"filebrowser/filebrowser:latest@sha256:86e8449ff8ff481fa6ca555d9f8ddf9ea887a41483526e889fc9b7d79b15d3a4\", falling back to regular pull." container=/tools_filebrowser.1.8uw2ng9td94zthwejvujv8x40 image="filebrowser/filebrowser:latest@sha256:86e8449ff8ff481fa6ca555d9f8ddf9ea887a41483526e889fc9b7d79b15d3a4"
time="2023-04-10T12:11:19Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_filebrowser.1.8uw2ng9td94zthwejvujv8x40 image="filebrowser/filebrowser:latest@sha256:86e8449ff8ff481fa6ca555d9f8ddf9ea887a41483526e889fc9b7d79b15d3a4"
time="2023-04-10T12:11:19Z" level=warning msg="Could not do a head request for \"inspircd/inspircd-docker:latest@sha256:3c9dfbc8a528341845b05275ff6f4376971ac07c39913bed27d645bb2ccee2db\", falling back to regular pull." container=/comms_inspircd.1.32z6b7loo5gg04ez3gdgo99gv image="inspircd/inspircd-docker:latest@sha256:3c9dfbc8a528341845b05275ff6f4376971ac07c39913bed27d645bb2ccee2db"
time="2023-04-10T12:11:19Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_inspircd.1.32z6b7loo5gg04ez3gdgo99gv image="inspircd/inspircd-docker:latest@sha256:3c9dfbc8a528341845b05275ff6f4376971ac07c39913bed27d645bb2ccee2db"
time="2023-04-10T12:11:19Z" level=warning msg="Could not do a head request for \"machines/filestash:latest@sha256:479c60fb2319e4a847516a0c40a700d567c336aee267a3210f565914750b58eb\", falling back to regular pull." container=/tools_filestash.1.fa0oq6c73yssb7ep7s4dj8g2j image="machines/filestash:latest@sha256:479c60fb2319e4a847516a0c40a700d567c336aee267a3210f565914750b58eb"
time="2023-04-10T12:11:19Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_filestash.1.fa0oq6c73yssb7ep7s4dj8g2j image="machines/filestash:latest@sha256:479c60fb2319e4a847516a0c40a700d567c336aee267a3210f565914750b58eb"
time="2023-04-10T12:11:20Z" level=warning msg="Could not do a head request for \"goofball222/murmur:latest@sha256:2e6f7cadcb2f0b700afc96475e5d5fc1567af784654f1a2bd9a5a62cbc1eb2d2\", falling back to regular pull." container=/comms_murmur.1.o0gz95tobon8qd1ejwtaeseli image="goofball222/murmur:latest@sha256:2e6f7cadcb2f0b700afc96475e5d5fc1567af784654f1a2bd9a5a62cbc1eb2d2"
time="2023-04-10T12:11:20Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_murmur.1.o0gz95tobon8qd1ejwtaeseli image="goofball222/murmur:latest@sha256:2e6f7cadcb2f0b700afc96475e5d5fc1567af784654f1a2bd9a5a62cbc1eb2d2"
time="2023-04-10T12:11:20Z" level=warning msg="Could not do a head request for \"zabbix/zabbix-server-pgsql:latest@sha256:1fd82373eab1f9430b63e0caeb5db758879eb8de6706e04b330f6dbea732fe0d\", falling back to regular pull." container=/sysmon_zabbix-server.1.nun4ukbr3wydlngpwb6d4dbh9 image="zabbix/zabbix-server-pgsql:latest@sha256:1fd82373eab1f9430b63e0caeb5db758879eb8de6706e04b330f6dbea732fe0d"
time="2023-04-10T12:11:20Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmon_zabbix-server.1.nun4ukbr3wydlngpwb6d4dbh9 image="zabbix/zabbix-server-pgsql:latest@sha256:1fd82373eab1f9430b63e0caeb5db758879eb8de6706e04b330f6dbea732fe0d"
time="2023-04-10T12:11:20Z" level=warning msg="Could not do a head request for \"orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8\", falling back to regular pull." container=/tools_orchardcms.1.w9ek8asx34to67juvhsh106x4 image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:20Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_orchardcms.1.w9ek8asx34to67juvhsh106x4 image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:20Z" level=warning msg="Could not do a head request for \"lissy93/dashy:latest@sha256:ce64e0c70855638e6eb91edba29baa2e2a294075eb0c81930311aa13cf08724b\", falling back to regular pull." container=/tools_dashy.1.z91ajjzt8u8sjgmvnf6fl118d image="lissy93/dashy:latest@sha256:ce64e0c70855638e6eb91edba29baa2e2a294075eb0c81930311aa13cf08724b"
time="2023-04-10T12:11:20Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_dashy.1.z91ajjzt8u8sjgmvnf6fl118d image="lissy93/dashy:latest@sha256:ce64e0c70855638e6eb91edba29baa2e2a294075eb0c81930311aa13cf08724b"
time="2023-04-10T12:11:21Z" level=warning msg="Could not do a head request for \"archivebox/archivebox:latest@sha256:3af58cc27a40ec3e18765c7d477f5fde3dfc149c5d8ba1d72545baf1e983fed3\", falling back to regular pull." container=/tools_archivebox.1.50wp5ujv6jlxz1jw3d24gvn8g image="archivebox/archivebox:latest@sha256:3af58cc27a40ec3e18765c7d477f5fde3dfc149c5d8ba1d72545baf1e983fed3"
time="2023-04-10T12:11:21Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_archivebox.1.50wp5ujv6jlxz1jw3d24gvn8g image="archivebox/archivebox:latest@sha256:3af58cc27a40ec3e18765c7d477f5fde3dfc149c5d8ba1d72545baf1e983fed3"
time="2023-04-10T12:11:21Z" level=warning msg="Could not do a head request for \"mbentley/omada-controller:latest@sha256:d3e9b9bb66d8d4fe7fdb431080606cd60a8a319240ed7fd23dbcf59065ca9ee7\", falling back to regular pull." container=/tools_omada.1.kt0qbre1vnhurwibz774m50ey image="mbentley/omada-controller:latest@sha256:d3e9b9bb66d8d4fe7fdb431080606cd60a8a319240ed7fd23dbcf59065ca9ee7"
time="2023-04-10T12:11:21Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_omada.1.kt0qbre1vnhurwibz774m50ey image="mbentley/omada-controller:latest@sha256:d3e9b9bb66d8d4fe7fdb431080606cd60a8a319240ed7fd23dbcf59065ca9ee7"
time="2023-04-10T12:11:21Z" level=warning msg="Could not do a head request for \"bitnami/keycloak:latest@sha256:eb146198a82d2092c10795404218ae5899baa9c7335b6ee290bfaad521a95f0a\", falling back to regular pull." container=/secur_keycloak.1.ot6ldx3z89t94hw2d6hnxcbhi image="bitnami/keycloak:latest@sha256:eb146198a82d2092c10795404218ae5899baa9c7335b6ee290bfaad521a95f0a"
time="2023-04-10T12:11:21Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/secur_keycloak.1.ot6ldx3z89t94hw2d6hnxcbhi image="bitnami/keycloak:latest@sha256:eb146198a82d2092c10795404218ae5899baa9c7335b6ee290bfaad521a95f0a"
time="2023-04-10T12:11:21Z" level=info msg="Session done" Failed=0 Scanned=18 Updated=0 notify=no
time="2023-04-10T12:11:47Z" level=warning msg="Could not do a head request for \"containrrr/watchtower:latest@sha256:0ca7a88fd0748aa6f32e50b67eb11148cdb989fc595264c2778c85297a2c1abe\", falling back to regular pull." container=/sysmaint_watchtower.1.4sko7jbkhkapl2ttgmigrfw1j image="containrrr/watchtower:latest@sha256:0ca7a88fd0748aa6f32e50b67eb11148cdb989fc595264c2778c85297a2c1abe"
time="2023-04-10T12:11:47Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmaint_watchtower.1.4sko7jbkhkapl2ttgmigrfw1j image="containrrr/watchtower:latest@sha256:0ca7a88fd0748aa6f32e50b67eb11148cdb989fc595264c2778c85297a2c1abe"
time="2023-04-10T12:11:47Z" level=warning msg="Could not do a head request for \"mietzen/bastillion:3.14.0@sha256:51a2641dac3d5fbbc4570ff48641437b54fa48ae6cf2f122439960f6032b8c21\", falling back to regular pull." container=/sysmaint_bastillion.1.74zmrju0y46f1zg5fve8ah0ko image="mietzen/bastillion:3.14.0@sha256:51a2641dac3d5fbbc4570ff48641437b54fa48ae6cf2f122439960f6032b8c21"
time="2023-04-10T12:11:47Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmaint_bastillion.1.74zmrju0y46f1zg5fve8ah0ko image="mietzen/bastillion:3.14.0@sha256:51a2641dac3d5fbbc4570ff48641437b54fa48ae6cf2f122439960f6032b8c21"
time="2023-04-10T12:11:47Z" level=warning msg="Could not do a head request for \"portainer/agent:latest@sha256:f763a45e6033b67b142d8dd8c5a4c450a514df2ea2bc6bffce8d671abd2341b8\", falling back to regular pull." container=/portainer_agent.s7av0vo9mflevxvx6jfk70tb0.mj2ldpaux5om1u4nbznir6mv9 image="portainer/agent:latest@sha256:f763a45e6033b67b142d8dd8c5a4c450a514df2ea2bc6bffce8d671abd2341b8"
time="2023-04-10T12:11:47Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/portainer_agent.s7av0vo9mflevxvx6jfk70tb0.mj2ldpaux5om1u4nbznir6mv9 image="portainer/agent:latest@sha256:f763a45e6033b67b142d8dd8c5a4c450a514df2ea2bc6bffce8d671abd2341b8"
time="2023-04-10T12:11:48Z" level=warning msg="Could not do a head request for \"portainer/portainer-ce:latest@sha256:9fa1ec78b4e29d83593cf9720674b72829c9cdc0db7083a962bc30e64e27f64e\", falling back to regular pull." container=/portainer_portainer.1.5mzsv9cqxn9ohcu0lcqqdebra image="portainer/portainer-ce:latest@sha256:9fa1ec78b4e29d83593cf9720674b72829c9cdc0db7083a962bc30e64e27f64e"
time="2023-04-10T12:11:48Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/portainer_portainer.1.5mzsv9cqxn9ohcu0lcqqdebra image="portainer/portainer-ce:latest@sha256:9fa1ec78b4e29d83593cf9720674b72829c9cdc0db7083a962bc30e64e27f64e"
time="2023-04-10T12:11:48Z" level=warning msg="Could not do a head request for \"zabbix/zabbix-web-nginx-pgsql:latest@sha256:3698815efa78501143bb9be06c4c06289242ca6fcfb682c1ebdb1325fb2a4441\", falling back to regular pull." container=/sysmon_zabbix-web.1.d0ogoi6p39n4uva8s3tls9nld image="zabbix/zabbix-web-nginx-pgsql:latest@sha256:3698815efa78501143bb9be06c4c06289242ca6fcfb682c1ebdb1325fb2a4441"
time="2023-04-10T12:11:48Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmon_zabbix-web.1.d0ogoi6p39n4uva8s3tls9nld image="zabbix/zabbix-web-nginx-pgsql:latest@sha256:3698815efa78501143bb9be06c4c06289242ca6fcfb682c1ebdb1325fb2a4441"
time="2023-04-10T12:11:48Z" level=warning msg="Could not do a head request for \"orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8\", falling back to regular pull." container=/tools_intranetcms.1.l8z3qv0axj0lrn4b1ottubzmh image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:48Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_intranetcms.1.l8z3qv0axj0lrn4b1ottubzmh image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:48Z" level=warning msg="Could not do a head request for \"teamspeak:latest@sha256:43ae19ea49a5af4870beb62f249d9dfa64a9a4e5d66a6bf421b7986cbd03d142\", falling back to regular pull." container=/comms_teamspeak.1.q408s3o7eucltpwqm1yektzy7 image="teamspeak:latest@sha256:43ae19ea49a5af4870beb62f249d9dfa64a9a4e5d66a6bf421b7986cbd03d142"
time="2023-04-10T12:11:48Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_teamspeak.1.q408s3o7eucltpwqm1yektzy7 image="teamspeak:latest@sha256:43ae19ea49a5af4870beb62f249d9dfa64a9a4e5d66a6bf421b7986cbd03d142"
time="2023-04-10T12:11:49Z" level=warning msg="Could not do a head request for \"thelounge/thelounge:latest@sha256:16d6203694c029db77a5ddb2d81ddc741977329268de17fe8b51b1363e7ee952\", falling back to regular pull." container=/comms_thelounge.1.zlol0hnjro2fis191td950isb image="thelounge/thelounge:latest@sha256:16d6203694c029db77a5ddb2d81ddc741977329268de17fe8b51b1363e7ee952"
time="2023-04-10T12:11:49Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_thelounge.1.zlol0hnjro2fis191td950isb image="thelounge/thelounge:latest@sha256:16d6203694c029db77a5ddb2d81ddc741977329268de17fe8b51b1363e7ee952"
time="2023-04-10T12:11:49Z" level=warning msg="Could not do a head request for \"filebrowser/filebrowser:latest@sha256:86e8449ff8ff481fa6ca555d9f8ddf9ea887a41483526e889fc9b7d79b15d3a4\", falling back to regular pull." container=/tools_filebrowser.1.8uw2ng9td94zthwejvujv8x40 image="filebrowser/filebrowser:latest@sha256:86e8449ff8ff481fa6ca555d9f8ddf9ea887a41483526e889fc9b7d79b15d3a4"
time="2023-04-10T12:11:49Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_filebrowser.1.8uw2ng9td94zthwejvujv8x40 image="filebrowser/filebrowser:latest@sha256:86e8449ff8ff481fa6ca555d9f8ddf9ea887a41483526e889fc9b7d79b15d3a4"
time="2023-04-10T12:11:49Z" level=warning msg="Could not do a head request for \"inspircd/inspircd-docker:latest@sha256:3c9dfbc8a528341845b05275ff6f4376971ac07c39913bed27d645bb2ccee2db\", falling back to regular pull." container=/comms_inspircd.1.32z6b7loo5gg04ez3gdgo99gv image="inspircd/inspircd-docker:latest@sha256:3c9dfbc8a528341845b05275ff6f4376971ac07c39913bed27d645bb2ccee2db"
time="2023-04-10T12:11:49Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_inspircd.1.32z6b7loo5gg04ez3gdgo99gv image="inspircd/inspircd-docker:latest@sha256:3c9dfbc8a528341845b05275ff6f4376971ac07c39913bed27d645bb2ccee2db"
time="2023-04-10T12:11:49Z" level=warning msg="Could not do a head request for \"machines/filestash:latest@sha256:479c60fb2319e4a847516a0c40a700d567c336aee267a3210f565914750b58eb\", falling back to regular pull." container=/tools_filestash.1.fa0oq6c73yssb7ep7s4dj8g2j image="machines/filestash:latest@sha256:479c60fb2319e4a847516a0c40a700d567c336aee267a3210f565914750b58eb"
time="2023-04-10T12:11:49Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_filestash.1.fa0oq6c73yssb7ep7s4dj8g2j image="machines/filestash:latest@sha256:479c60fb2319e4a847516a0c40a700d567c336aee267a3210f565914750b58eb"
time="2023-04-10T12:11:50Z" level=warning msg="Could not do a head request for \"goofball222/murmur:latest@sha256:2e6f7cadcb2f0b700afc96475e5d5fc1567af784654f1a2bd9a5a62cbc1eb2d2\", falling back to regular pull." container=/comms_murmur.1.o0gz95tobon8qd1ejwtaeseli image="goofball222/murmur:latest@sha256:2e6f7cadcb2f0b700afc96475e5d5fc1567af784654f1a2bd9a5a62cbc1eb2d2"
time="2023-04-10T12:11:50Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/comms_murmur.1.o0gz95tobon8qd1ejwtaeseli image="goofball222/murmur:latest@sha256:2e6f7cadcb2f0b700afc96475e5d5fc1567af784654f1a2bd9a5a62cbc1eb2d2"
time="2023-04-10T12:11:50Z" level=warning msg="Could not do a head request for \"zabbix/zabbix-server-pgsql:latest@sha256:1fd82373eab1f9430b63e0caeb5db758879eb8de6706e04b330f6dbea732fe0d\", falling back to regular pull." container=/sysmon_zabbix-server.1.nun4ukbr3wydlngpwb6d4dbh9 image="zabbix/zabbix-server-pgsql:latest@sha256:1fd82373eab1f9430b63e0caeb5db758879eb8de6706e04b330f6dbea732fe0d"
time="2023-04-10T12:11:50Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/sysmon_zabbix-server.1.nun4ukbr3wydlngpwb6d4dbh9 image="zabbix/zabbix-server-pgsql:latest@sha256:1fd82373eab1f9430b63e0caeb5db758879eb8de6706e04b330f6dbea732fe0d"
time="2023-04-10T12:11:50Z" level=warning msg="Could not do a head request for \"orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8\", falling back to regular pull." container=/tools_orchardcms.1.w9ek8asx34to67juvhsh106x4 image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:50Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_orchardcms.1.w9ek8asx34to67juvhsh106x4 image="orchardproject/orchardcore-cms-linux:latest@sha256:97a741321e2995cbe1432ac35cfb916be220df3e5840f17ff2b4add35a1559c8"
time="2023-04-10T12:11:50Z" level=warning msg="Could not do a head request for \"lissy93/dashy:latest@sha256:ce64e0c70855638e6eb91edba29baa2e2a294075eb0c81930311aa13cf08724b\", falling back to regular pull." container=/tools_dashy.1.z91ajjzt8u8sjgmvnf6fl118d image="lissy93/dashy:latest@sha256:ce64e0c70855638e6eb91edba29baa2e2a294075eb0c81930311aa13cf08724b"
time="2023-04-10T12:11:50Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_dashy.1.z91ajjzt8u8sjgmvnf6fl118d image="lissy93/dashy:latest@sha256:ce64e0c70855638e6eb91edba29baa2e2a294075eb0c81930311aa13cf08724b"
time="2023-04-10T12:11:51Z" level=warning msg="Could not do a head request for \"archivebox/archivebox:latest@sha256:3af58cc27a40ec3e18765c7d477f5fde3dfc149c5d8ba1d72545baf1e983fed3\", falling back to regular pull." container=/tools_archivebox.1.50wp5ujv6jlxz1jw3d24gvn8g image="archivebox/archivebox:latest@sha256:3af58cc27a40ec3e18765c7d477f5fde3dfc149c5d8ba1d72545baf1e983fed3"
time="2023-04-10T12:11:51Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_archivebox.1.50wp5ujv6jlxz1jw3d24gvn8g image="archivebox/archivebox:latest@sha256:3af58cc27a40ec3e18765c7d477f5fde3dfc149c5d8ba1d72545baf1e983fed3"
time="2023-04-10T12:11:51Z" level=warning msg="Could not do a head request for \"mbentley/omada-controller:latest@sha256:d3e9b9bb66d8d4fe7fdb431080606cd60a8a319240ed7fd23dbcf59065ca9ee7\", falling back to regular pull." container=/tools_omada.1.kt0qbre1vnhurwibz774m50ey image="mbentley/omada-controller:latest@sha256:d3e9b9bb66d8d4fe7fdb431080606cd60a8a319240ed7fd23dbcf59065ca9ee7"
time="2023-04-10T12:11:51Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/tools_omada.1.kt0qbre1vnhurwibz774m50ey image="mbentley/omada-controller:latest@sha256:d3e9b9bb66d8d4fe7fdb431080606cd60a8a319240ed7fd23dbcf59065ca9ee7"
time="2023-04-10T12:11:51Z" level=warning msg="Could not do a head request for \"bitnami/keycloak:latest@sha256:eb146198a82d2092c10795404218ae5899baa9c7335b6ee290bfaad521a95f0a\", falling back to regular pull." container=/secur_keycloak.1.ot6ldx3z89t94hw2d6hnxcbhi image="bitnami/keycloak:latest@sha256:eb146198a82d2092c10795404218ae5899baa9c7335b6ee290bfaad521a95f0a"
time="2023-04-10T12:11:51Z" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/secur_keycloak.1.ot6ldx3z89t94hw2d6hnxcbhi image="bitnami/keycloak:latest@sha256:eb146198a82d2092c10795404218ae5899baa9c7335b6ee290bfaad521a95f0a"
time="2023-04-10T12:11:51Z" level=info msg="Session done" Failed=0 Scanned=18 Updated=0 notify=no

I get this for every container.

piksel commented 1 year ago

@tessierp you have incorrect yaml syntax, that's why the ignore doesn't work (remove the "s or use the dict syntax instead of array syntax). That only hides the problem though, you are using pinned images which watchtower never updates (since pinned means "explicitly use this exact version of the image"). But as I said, create a new discussion in QnA if you need assistance, this is not related to this issue.

tessierp commented 1 year ago

@tessierp you have incorrect yaml syntax, that's why the ignore doesn't work (remove the "s or use the dict syntax instead of array syntax). That only hides the problem though, you are using pinned images which watchtower never updates (since pinned means "explicitly use this exact version of the image"). But as I said, create a new discussion in QnA if you need assistance, this is not related to this issue.

Thanks, I will do that.. However, the syntax is good, the container deploys and runs. But it will not update due to the "auth" it looks like. Also, yes I do use "latest" for all my images. So what you are saying is I would need to remove the "latest" tag?

Anyway will create a new issue. Thanks