search

pygmystack / pygmy

the pygmy stack is a container stack for local development
MIT License
25 stars 12 forks source link

#478: Add update loop for uselagoon/* #479

Closed fubarhouse closed 10 months ago

fubarhouse commented 11 months ago

Resolves #478 

~/GolandProjects/pygmy $ go run main.go pull
Using config file: /home/karl/.pygmy.yml
Image docker.io/pygmystack/ssh-agent:latest is up to date
Image docker.io/pygmystack/dnsmasq:latest is up to date
Image docker.io/pygmystack/mailhog:latest is up to date
Image docker.io/pygmystack/haproxy:latest is up to date
Image docker.io/uselagoon/remote-controller:latest is up to date
Image docker.io/uselagoon/database-tools:latest is up to date
Image docker.io/uselagoon/lagoon-cli:latest is up to date
Image docker.io/uselagoon/php-8.2-cli-drupal:latest is up to date
Image docker.io/uselagoon/php-8.1-cli-drupal:latest is up to date
Image docker.io/uselagoon/php-8.2-cli:latest is up to date
Image docker.io/uselagoon/php-8.1-cli:latest is up to date
Image docker.io/uselagoon/php-8.0-cli-drupal:latest is up to date
Image docker.io/uselagoon/php-8.0-cli:latest is up to date
Image docker.io/uselagoon/ruby-3.2:latest is up to date
Image docker.io/uselagoon/ruby-3.1:latest is up to date
Image docker.io/uselagoon/ruby-3.0:latest is up to date
Image docker.io/uselagoon/mariadb-10.6:latest is up to date
Image docker.io/uselagoon/varnish-7:latest is up to date
Image docker.io/uselagoon/php-8.0-fpm:latest is up to date
Image docker.io/uselagoon/php-8.2-fpm:latest is up to date
Image docker.io/uselagoon/php-8.1-fpm:latest is up to date
Image docker.io/uselagoon/ui:latest is up to date
Image docker.io/uselagoon/build-deploy-image:latest is up to date
Image docker.io/uselagoon/insights-handler:latest is up to date
Image docker.io/uselagoon/docker-host:latest is up to date
Image docker.io/uselagoon/logs-dispatcher:latest is up to date
Image docker.io/uselagoon/logs-concentrator:latest is up to date
Image docker.io/uselagoon/drush-alias:latest is up to date
Image docker.io/uselagoon/athenapdf-service:latest is up to date
~/GolandProjects/pygmy $
rocketeerbkw commented 11 months ago

Built and ran locally, I have identified two issues.

Background

➜  pygmy git:(issue/478) ./builds/pygmy-linux-amd64-static update
Image docker.io/pygmystack/ssh-agent:latest is up to date
Image docker.io/pygmystack/dnsmasq:latest is up to date
Image docker.io/pygmystack/haproxy:latest is up to date
Image docker.io/pygmystack/mailhog:latest is up to date
Image docker.io/uselagoon/nginx-drupal:latest is up to date
Image docker.io/uselagoon/ui:latest is up to date
Successfully pulled docker.io/uselagoon/solr-8-drupal:latest
Successfully pulled docker.io/uselagoon/solr-7-drupal:latest
Successfully pulled docker.io/uselagoon/solr-8:latest
Successfully pulled ghcr.io/uselagoon/build-deploy-image:latest
Successfully pulled docker.io/uselagoon/redis-6:latest
Successfully pulled docker.io/uselagoon/nginx:latest
Image docker.io/uselagoon/solr-8:latest is up to date
Image docker.io/uselagoon/nginx:latest is up to date
Successfully pulled docker.io/uselagoon/node-16-builder:latest
Image docker.io/uselagoon/nginx-drupal:latest is up to date
Successfully pulled docker.io/uselagoon/redis-5:latest
Successfully pulled docker.io/uselagoon/postgres-15:latest
Successfully pulled docker.io/uselagoon/node-16:latest
Successfully pulled docker.io/uselagoon/commons:latest

These were my local images BEFORE

➜  ~ docker images | grep uselagoon
uselagoon/php-8.1-cli-drupal            latest    abe942ea3e08   13 days ago      560MB
uselagoon/php-8.1-cli                   23.8.0    ff61f20ee5a5   13 days ago      551MB
uselagoon/php-8.1-cli                   latest    ff61f20ee5a5   13 days ago      551MB
uselagoon/nginx-drupal                  latest    298c614720a2   2 weeks ago      139MB
uselagoon/mariadb-10.5-drupal           latest    f4ca1a77d3a0   2 weeks ago      256MB
uselagoon/php-8.1-fpm                   latest    15dbdb41d237   2 weeks ago      306MB
uselagoon/ui                            latest    8449595e1906   6 weeks ago      789MB
uselagoon/solr-8-drupal                 latest    7118f06ba545   8 weeks ago      537MB
uselagoon/solr-7-drupal                 latest    ac771ae4f275   8 weeks ago      466MB
uselagoon/mariadb-10.5-drupal           <none>    57551d555870   8 weeks ago      256MB
uselagoon/solr-8                        latest    01d254a15c0b   8 weeks ago      537MB
ghcr.io/uselagoon/build-deploy-image    pr-204    9475aa540608   4 months ago     497MB
uselagoon/php-8.2-cli                   latest    84c2ec5a074b   4 months ago     513MB
uselagoon/mariadb-10.6-drupal           latest    9c6c6fa00e15   4 months ago     261MB
uselagoon/redis-6                       latest    8fcd32a5ebe4   4 months ago     41.4MB
uselagoon/nginx                         latest    c48f16bfd9d5   4 months ago     133MB
uselagoon/php-8.2-cli-drupal            latest    154626772ba9   5 months ago     519MB
uselagoon/mariadb-10.4                  latest    a715747f03f4   5 months ago     243MB
uselagoon/solr-8                        <none>    86223e0c0b58   5 months ago     538MB
uselagoon/php-8.2-fpm                   latest    b96df0ca5c67   5 months ago     314MB
uselagoon/nginx                         <none>    54d2bd0b19dc   5 months ago     133MB
uselagoon/php-8.1-cli-drupal            <none>    ae8d549e1767   7 months ago     509MB
uselagoon/node-16-builder               latest    fe95dd438782   7 months ago     450MB
uselagoon/nginx-drupal                  <none>    fec1a154dec6   7 months ago     115MB
uselagoon/redis-5                       latest    b0bd79b425d0   7 months ago     36.2MB
uselagoon/mariadb-10.6                  latest    39901b190e9b   7 months ago     260MB
uselagoon/mariadb-10.6-drupal           <none>    a1fa2f509e53   7 months ago     260MB
uselagoon/postgres-15                   latest    49c4672fbb09   7 months ago     256MB
uselagoon/php-8.1-fpm                   <none>    fa20927816fd   7 months ago     306MB
uselagoon/node-16                       latest    57b46fc3285f   7 months ago     131MB
uselagoon/python-3.10                   latest    891e0413aa85   7 months ago     100MB
uselagoon/commons                       latest    ddb90e9f39b8   7 months ago     22.6MB

Issue 1

A number of images were not pulled (or attempted to pull). If we only look at php 8.2 as an example, there is the cli, cli-drupal, and fpm images. My local uselagoon/php-8.2-cli:latest AFTER the update is still four months old but the most recent one is two weeks old. It's the same for cli-drupal and fpm.

It looks to me like all the images with a dot separated version didn't get pulled. Here's the list of skipped images:

uselagoon/php-8.1-cli-drupal
uselagoon/php-8.1-cli
uselagoon/php-8.1-cli
uselagoon/mariadb-10.5-drupal
uselagoon/php-8.1-fpm
uselagoon/mariadb-10.5-drupal
uselagoon/php-8.2-cli
uselagoon/mariadb-10.6-drupal
uselagoon/php-8.2-cli-drupal
uselagoon/mariadb-10.4
uselagoon/php-8.2-fpm
uselagoon/php-8.1-cli-drupal
uselagoon/mariadb-10.6
uselagoon/mariadb-10.6-drupal
uselagoon/php-8.1-fpm
uselagoon/python-3.10

Issue 2

In the first run, I noticed that uselagoon/nginx-drupal was pulled twice. I re-ran the update after getting all the info for issue 1 and noticed a lot more duplicate pulls now. I think it's trying to pull all images regardless of the tag, which means all the dangling images from the first update now cause another pull attempt on the next update. It doesn't hurt anything, but it might be confusing to the user if the output ends up being really long and there are multiple pull attempts per image.

It's probably correct to skip dangling images? In the docker cli these are shown as the <none> tag. I also have a uselagoon/php-8.1-cli:23.8.0 image locally, and even though it wasn't pulled for some reason (see issue 1), it probably doesn't need to be pulled at all? Maybe we change the logic to only search for latest tagged images locally which would also solve the dangling images duplicates?

Second run:

➜  pygmy git:(patch-1) ./builds/pygmy-linux-amd64-static update
Image docker.io/pygmystack/haproxy:latest is up to date
Image docker.io/pygmystack/mailhog:latest is up to date
Image docker.io/pygmystack/ssh-agent:latest is up to date
Image docker.io/pygmystack/dnsmasq:latest is up to date
Image docker.io/uselagoon/solr-8-drupal:latest is up to date
Image docker.io/uselagoon/solr-7-drupal:latest is up to date
Image docker.io/uselagoon/node-16-builder:latest is up to date
Image docker.io/uselagoon/nginx-drupal:latest is up to date
Image docker.io/uselagoon/redis-5:latest is up to date
Image docker.io/uselagoon/redis-6:latest is up to date
Image docker.io/uselagoon/postgres-15:latest is up to date
Image docker.io/uselagoon/solr-8:latest is up to date
Image docker.io/uselagoon/node-16:latest is up to date
Image docker.io/uselagoon/nginx:latest is up to date
Image docker.io/uselagoon/commons:latest is up to date
Image docker.io/uselagoon/ui:latest is up to date
Image ghcr.io/uselagoon/build-deploy-image:latest is up to date
Image docker.io/uselagoon/solr-8-drupal:latest is up to date
Image docker.io/uselagoon/solr-7-drupal:latest is up to date
Image docker.io/uselagoon/solr-8:latest is up to date
Image ghcr.io/uselagoon/build-deploy-image:latest is up to date
Image docker.io/uselagoon/redis-6:latest is up to date
Image docker.io/uselagoon/nginx:latest is up to date
Image docker.io/uselagoon/solr-8:latest is up to date
Image docker.io/uselagoon/nginx:latest is up to date
Image docker.io/uselagoon/node-16-builder:latest is up to date
Image docker.io/uselagoon/nginx-drupal:latest is up to date
Image docker.io/uselagoon/redis-5:latest is up to date
Image docker.io/uselagoon/postgres-15:latest is up to date
Image docker.io/uselagoon/node-16:latest is up to date
Image docker.io/uselagoon/commons:latest is up to date
rocketeerbkw commented 11 months ago

All the images I have locally attempted an update, so I think this is good to go, thanks!