DockerCompose: force recreate images

Implementation proposal

https://github.com/indigo-dc/jenkins-pipeline-library/blob/35fc9402dc9098d3692e11448898b69fac1b1443/src/eu/indigo/compose/DockerCompose.groovy#L95 Images purge is implemented but not available to be used as an available property. https://github.com/indigo-dc/jenkins-pipeline-library/blob/35fc9402dc9098d3692e11448898b69fac1b1443/src/eu/indigo/compose/DockerCompose.groovy#L206

This can become a feature to be available from the container options:

container_groups:
  groupA:
   services:
   - service1
   - service2
   options:
    ignore_push_failures: 'false'
    force_docker_build: 'false'
    purge_images: 'false'

purge_images will only be run in the end when defining cleanup: true:

qc_delivery:
  repos:
   repoA:
    deployment:
     name: asdfasdfa1
     cleanup: true
    push: groupA
    keepgoing: true

so that when running docker-compose down it will select the command to do the complete purge of the deployment images, volumes and containers.

purge_images parameter will only be valid to docker-compose composer. This will have no action when a deployment is using kubernetes composer. The configuration for container_groups is generic and was intentionally left out of deployment, since the properties defined in that scope covers all possible features for container images that MAY be implemented by the composer.

Some additional information

For docker-compose up, the parameter --force-recreate only works if image is being built locally. This is supported in JePL with forceBuild option with --build flag. For the new release can be enabled with force_docker_build:

container_groups:
  groupA:
   services:
   - service1
   - service2
   options:
    ignore_push_failures: 'false'
    force_docker_build: 'false'
    purge_images: 'false'

pull_policy: always in compose configuration file didn't pull an image if it already exists in cache. Quoting the documentation:

If the image does not exist on the platform, Compose implementations MUST attempt to pull it based on the pull_policy. Compose implementations with build support MAY offer alternative options for the end user to control precedence of pull over building the image from source, however pulling the image MUST be the default behavior.

Also is not clear the difference between the expected result between the pull policy always and missing, from the previous definition:

always: Compose implementations SHOULD always pull the image from the registry. missing: Compose implementations SHOULD pull the image only if it's not available in the platform cache. This SHOULD be the default option for Compose implementations without build support. if_not_present SHOULD be considered an alias for this value for backward compatibility.

As a conclusion of my experiences, maybe docker-compose only looks into the docker image tag. When reviewing the behavior of missing option, if an image tag points to an existing image in the cache it will only create the required tag. Changing to always docker-compose should always pull from registry for all new tags. But the cache is always used if docker-compose finds that the image already exists, as when running docker pull.

Kubernetes

With kubernetes the behavior is exactly the same: https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy

But in this case will be even more important that used images are the released images, because there is no implementation to clean the cache during a deployment (this is done automatically by the garbage collector of kubernetes from the defined thresholds for the eviction). My advise is to always rely on real container images releases, avoiding kubernetes to pull images that are experimental and can be replaced within the same tag. All tests to build the images for a release must be run in docker-compose composer that allows to do a complete purge and start from scratch, like in the developer workstation.

indigo-dc / jenkins-pipeline-library