concourse / registry-image-resource

a resource for images in a Docker registry
Apache License 2.0
89 stars 107 forks source link

Allow overriding Repository / Tag on Get #337

Open adnankobir opened 1 year ago

adnankobir commented 1 year ago

This is a very simplistic approach to a specific use-case:

We have various docker base images that we would like to pull in to build from.

We currently use the registry-image-resource to pull this base image in oci format to satisfy the FROM requirement in our Dockerfiles from a private repository using ARG:

Dockerfile

ARG base_image=base
FROM ${base_image}

...

Pipeline

resources:
- base
   type: registry-image
   source:
     aws_access_key_id: ((id))
     aws_secret_access_key: ((key)
     aws_region: ((region))
     repository: base-dotnet
     tag: 1.0.0

jobs:
  ...
  # we trigger the pipeline some other way

  - get: base
     params:
       format: oci

  - task: build
     privileged: true
     config:
       image_resource:
         type: registry-image
         source:
            repository: concourse/oci-build-task
       inputs:
          - name: base
       ...

We now would like to parallelize our builds of multiple docker images - we produce a json that contains metadata about each of docker images we want to build - including the base we want to use - this is what requires us to override the source configuration for repository and tag:

metadata.json

[
  { 
    "service": "foo-api",
    "base_image": "foo-base",
    "base_tag": "1.0.0"
  },
  { 
    "service": "bar-api",
    "base_image": "bar-base",
    "base_tag": "2.0.0"
  }
]

Pipeline

jobs:
  ...
  # we trigger the pipeline some other way

  - load_var: metadata
    file: metadata.json

  - do:
    - get: base
       params:
         format: oci
         repository: ((.:service.base_image))
         tag: ((.:service.base_tag))

    - task: build
       privileged: true
       config:
         image_resource:
           type: registry-image
           source:
              repository: concourse/oci-build-task
         inputs:
            - name: base
       ...

     across:
       - var: service
          max_in_fligh: all
          values: ((.:metadata))

Now clearly this will have implications if we wanted to use the base/registry-image-resource to trigger.

Any thoughts around a better way to achieve what we want? cc @taylorsilva