search

MichelFR / MqDockerUp

Mqtt Docker Updater (MqDockerUp) is a tool for updating Docker containers. It provides an easy way to check for updates and publish changes to a MQTT broker.
Apache License 2.0
55 stars 6 forks source link

MqDockerUp crashing frequently #267

Open realcanadrian opened 3 months ago

realcanadrian commented 3 months ago

I'm not quite sure why, but MqDockerUp keeps crashing. I keep noticing updates listed in Home Assistant that I have already installed, and when I check my docker containers on UnRAID, MqDockerUp has quit. The latest time was during some container updates. This is the end of the log when the container quit:

19.03.2024 13:04:38 [info]      Checking for removed containers...
19.03.2024 13:04:38 [info]      Checking for containers...
19.03.2024 13:04:38 [info]      Checking for image updates...
19.03.2024 13:04:38 [error]     MqDockerUp stopped due to an error
19.03.2024 13:04:38 [error]     object
19.03.2024 13:04:38 [error]     Error: (HTTP code 404) no such container - No such container: e119ba7d1906d6ea633d3c7d40353acf0dd2fe1cd744edc4538f05bd5633ba2b 
    at /app/node_modules/docker-modem/lib/modem.js:382:17
    at getCause (/app/node_modules/docker-modem/lib/modem.js:417:7)
    at Modem.buildPayload (/app/node_modules/docker-modem/lib/modem.js:378:5)
    at IncomingMessage.<anonymous> (/app/node_modules/docker-modem/lib/modem.js:346:16)
    at IncomingMessage.emit (node:events:529:35)
    at IncomingMessage.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

Based on the log I'm assuming MqDockerUp was running update checks still when one of the containers was upgraded, and it panicked because a container it expected to be there wasn't (because it was updated). Any ideas about whether MqDockerUp can recover gracefully from this rather than dying?

github-actions[bot] commented 3 months ago

👋 Welcome to the project! Thank you for raising your first issue. We will look into it as soon as possible.

realcanadrian commented 3 months ago

Oh, and right after I restarted the container:

> mqdockerup@1.8.0 start
> ts-node src/index.ts

  __  __       ____             _             _   _       
 |  \/  | __ _|  _ \  ___   ___| | _____ _ __| | | |_ __  
 | |\/| |/ _` | | | |/ _ \ / __| |/ / _ \ '__| | | | '_ \ 
 | |  | | (_| | |_| | (_) | (__|   <  __/ |  | |_| | |_) |
 |_|  |_|\__, |____/ \___/ \___|_|\_\___|_|   \___/| .__/ 
            |_|                                    |_|    
Starting MqDockerUp V1.8.0 ... 

19.03.2024 13:13:33 [info]      Connected to the database.
19.03.2024 13:13:33 [info]      Database initialized successfully
19.03.2024 13:13:33 [error]     MQTT client connection error:  connack timeout
19.03.2024 13:13:33 [error]     MqDockerUp stopped due to an error
19.03.2024 13:13:33 [error]     object
19.03.2024 13:13:33 [error]     Error: connack timeout
    at Timeout._onTimeout (/app/node_modules/mqtt/src/lib/client.ts:853:23)
    at listOnTimeout (node:internal/timers:569:17)
    at processTimers (node:internal/timers:512:7)

Maybe some "retry" logic to be a bit more resilient?

eekdood commented 3 months ago

Same issue here, although I haven't seen the "no such container" messages. Just continuous, every minute or so, "connack timeout" messages followed by the container restarting. Been working on it for a few days. Not sure when exactly it started.

NsmUnited commented 3 months ago

Same here on my second machine... the first one works perfectly

nanderson97651 commented 3 months ago

I also had the same issue with 'no such' available. Mine specifically was an esphome/esphome:latest image that was in a stopped container. I've since switched to a fixed tag for esphome from ghcr.io so it's possible the old one doesn't exist anymore.

I'll add that mine was an image that hadn't been used in over 10 months that only recently started showing in MqDockerUp. I'm not sure that I love the default behavior to check for all images on the machine. I'd prefer to only check running container images or even only images that have been used in the last 4 weeks by default.

Jejebond commented 1 month ago

Same issue here. When i call an update, the container stop. To resume : 

02.06.2024 05:25:46 [error]     object
02.06.2024 05:25:46 [error]     SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at IncomingMessage.<anonymous> (/app/src/services/DockerService.ts:23:28)
    at IncomingMessage.emit (node:events:517:28)
    at IncomingMessage.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    at IncomingMessage.Readable.push (node:internal/streams/readable:278:10)
    at HTTPParser.parserOnBody (node:_http_common:131:24)
    at Socket.socketOnData (node:_http_client:541:22)
    at Socket.emit (node:events:517:28)

And after it tries to restart withe the same error in loop a fue times (2 times).

02.06.2024 05:26:02 [error]     MQTT client connection error:  connack timeout
02.06.2024 05:26:02 [error]     MqDockerUp stopped due to an error
02.06.2024 05:26:02 [error]     object
02.06.2024 05:26:02 [error]     Error: connack timeout
    at Timeout._onTimeout (/app/node_modules/mqtt/src/lib/client.ts:853:23)
    at listOnTimeout (node:internal/timers:569:17)
    at processTimers (node:internal/timers:512:7)

It finally succeed on 3rd restart with 1 error.

02.06.2024 05:26:30 [info]  Connected to the database.
02.06.2024 05:26:30 [info]  MQTT client successfully connected
02.06.2024 05:26:30 [info]  Checking for removed containers...
02.06.2024 05:26:30 [info]  Database initialized successfully
02.06.2024 05:26:30 [info]  Checking for containers...
02.06.2024 05:26:31 [info]  Checking for image updates...
02.06.2024 05:26:31 [info]  Image nocturno/hyper-hdr:latest is up-to-date
02.06.2024 05:26:32 [info]  Image linuxserver/overseerr:latest is up-to-date
02.06.2024 05:26:32 [info]  New version available for image linuxserver/prowlarr:latest
02.06.2024 05:26:32 [info]  Image linuxserver/radarr:latest is up-to-date
02.06.2024 05:26:33 [info]  Image linuxserver/sonarr:latest is up-to-date
02.06.2024 05:26:33 [info]  Image micrib/mqdockerup:latest is up-to-date
02.06.2024 05:26:33 [info]  Image flaresolverr/flaresolverr:latest is up-to-date
02.06.2024 05:26:34 [info]  Image adguard/adguardhome:latest is up-to-date
02.06.2024 05:26:34 [info]  Image portainer/portainer-ce:latest is up-to-date
02.06.2024 05:26:36 [info]  Image jlesage/jdownloader-2:latest is up-to-date
**Failed to check for new Dockerhub image digest: TypeError: Cannot read properties of undefined (reading 'split')
02.06.2024 05:26:56 [error]     Cannot read properties of undefined (reading 'split')**
02.06.2024 05:26:56 [warn]  Failed to find new digest for image markusmcnugen/qbittorrentvpn:latest
02.06.2024 05:26:56 [info]  Image containrrr/watchtower:latest is up-to-date
02.06.2024 05:26:56 [info]  Finished checking for image updates
02.06.2024 05:26:56 [info]  Next check in 5 minutes

Docker sometimes crash during an update. Then, i have to manually update, restore and restart the container aimed by mqdockerup.