Docker Compose doesn't load relocated images, causing airgapped deployments to fail

[bherw]

According to https://porter.sh/distribute-bundles/#image-references-after-publishing, "Bundles should not hard-code image references and instead should use the images section and templating so that they are referencing the published location of the image."

However, a simple docker-compose.yaml like the following is hardcoding the image references since the porter templating does not apply to the docker-compose.yaml:

services:
  foo:
    image: alpine
    entrypoint: /bin/sh -c 'echo test'

This example fails when relocated into an airgapped environment even if the alpine image is specified in porter.yaml because this docker-compose.yaml file is copied into the invocation image verbatim. docker-compose just sees the "alpine" reference and tries to look it up on docker.io, so the docker-compose execution fails. The official Porter airgap example, on the other hand, works, because the helm command gets the templated image references in the porter.yaml.

We need a way to get the image references from porter to the docker-compose config, ideally without making it impossible to test the compose file simply in the local environment by running docker-compose up.

[bherw]

docker-compose does pick up environment variables, so what I'm doing right now to get this working is a bit of a hack. I have a script doing the following:

• read the docker-compose to find the list of images it needs by content digest
• rewrite porter.yaml with the image manifest
• add exec statements to porter.yaml which echo the image references from the manifest into a file inside the invocation image's container
• rewrite the compose file to use environment variables
• load that file into docker-compose

It ends up looking something like this:

porter.yaml

images:
  alpine:
    repository: alpine
    digest: alpine@sha256:abcd...........

install:
  - exec:
    command: /bin/sh -c 'echo SERVICE_FOO_IMAGE={{bundle.images.foo.repository}}@{{bundle.images.foo.digest}} >> /root/porter.env'
  - docker-compose:
    arguments:
      - --env-file
      - /root/porter.env
      - up
      - -d

docker-compose.yaml

services:
  foo:
    image: $SERVICE_FOO_IMAGE
    entrypoint: /bin/sh -c 'echo test'

Obviously this isn't ideal, but it does work. Along this line, perhaps the docker-compose mixin could produce a similar environment file from the images.

Another idea could be to modify the docker-compose.yaml file to accept input, perhaps from the environment, as a part of the invocation image build process.

[vdice]

Thanks for creating this issue and sharing your current approach @bherw. It would definitely be nice to add built-in support to this mixin for updating the images in the compose yaml. As you've mentioned, there are at least a few approaches we could use. I do like the suggestion of seeing if it can be done at the build stage.

[carolynvs]

What if we had the docker-compose mixin make the .env for you? It can't be a build time, because the location can change when the bundle is published.

When compose commands are called, the mixin can check if any images are defined in porter.yaml and write them out with their proper values from /cnab/app/relocation-mapping.json to an .env file. Then if we created an .env file, add --env-file argument and pass it into docker-compose when it's run.

That way the author doesn't really need to worry about how to make the bundle airgap ready, the mixin can just handle it.

[sathish-kumar-narayanan]

What if we had the docker-compose mixin make the .env for you? It can't be a build time, because the location can change when the bundle is published.

When compose commands are called, the mixin can check if any images are defined in porter.yaml and write them out with their proper values from /cnab/app/relocation-mapping.json to an .env file. Then if we created an .env file, add --env-file argument and pass it into docker-compose when it's run.

That way the author doesn't really need to worry about how to make the bundle airgap ready, the mixin can just handle it.

.env may have many user defined variables. Instead of creating .env, Can we edit the images tag in docker-compose.yaml directly. Like the way kustomize or helm handles it.

[carolynvs]

Yeah I think that would be possible too. So anywhere in the docker-compose.yaml, we would look for the old image reference and replace it with the relocated one? I'm just checking if there is more interesting logic than a blanket find/replace.

[sathish-kumar-narayanan]

The other option would be layering it with another compose yaml. Mixin can create one another compose yaml docker-compose.images.yaml with the list of images referenced in porter.yaml, and supply it as file argument in addition to the provided one

[bherw]

Oops, I forgot about this for a bit.

What if we had the docker-compose mixin make the .env for you? It can't be a build time, because the location can change when the bundle is published.

I concur with @sathish-kumar-narayanan, this risks overwriting/overriding a user-provided file. However, you can use any variable from the environment in your docker-compose file, so porter need only set the right environment variables if we use this approach. I just used the .env file because it was convenient for my use case and easier to manipulate using scripts in the install phase.

When compose commands are called, the mixin can check if any images are defined in porter.yaml and write them out with their proper values from /cnab/app/relocation-mapping.json to an .env file. Then if we created an .env file, add --env-file argument and pass it into docker-compose when it's run.

That way the author doesn't really need to worry about how to make the bundle airgap ready, the mixin can just handle it.

I think this is a step in the right direction, but I see a few potential problems:

If the author of the docker-compose.yaml wrote "ubuntu:20.04" or some such tag-based reference as their service image, how do we match it to the image in the porter.yaml? My wrapper scripts can make the match because they resolve the tag to a digest at build time and write the digest into both the porter.yaml and create an updated docker-compose.yaml. We could say that this feature only works with digest references to ensure the match, or perhaps require that the image name in porter.yaml match the service.image in docker-compose.yaml.

With this setup, the author of the bundle still needs to manually create the images configuration in the porter.yaml in order to make the bundle airgap-ready, which is a not insignificant amount of work potentially. There are two key components which we need for relocation: resolving the images in porter.yaml and supporting relocation by updating the references in docker-compose.yaml.

If porter provides the relocated paths using environment variables, the docker-compose.yaml author also needs to write the image references to use the variables as well as default to the plain path in case they want to test their docker-compose.yaml locally. For example: ${ALPINE_12:-alpine:12}

The other option would be layering it with another compose yaml. Mixin can create one another compose yaml docker-compose.images.yaml with the list of images referenced in porter.yaml, and supply it as file argument in addition to the provided one

This seems like a nicer integration than using environment variables since it doesn't require the bundle author to write their service image config in a special way.

[carolynvs]

I can't quite tell if you have a suggested solution or not yet? Here's some context extra context that may be helpful (or not):

• the images map in porter.yaml will support resolving the tag to a digest at buildtime https://github.com/getporter/porter/issues/1357. So if an author does have to add the images used, it's a little less work.
• I'd be up for injecting the mapped images as environment variables, e.g. PORTER_IMAGES_KEY=relocated-image-with-digest so something like PORTER_IMAGES_NGINX=localhost:5000/nginx@sha256:abc123

[sathish-kumar-narayanan]

injecting the mapped images as environment variables sounds like a good idea.

[bherw]

The images map supporting tag resolution will help a lot. I think that the relocation problem should be fully covered by that and @sathish-kumar-narayanan's suggestion to make docker-compose-mixin write and load a docker-compose.images.yaml at runtime that overrides the service images in the author's docker-compose.yaml.

It could be helpful if Porter injected the mapped images as environment variables, but it doesn't make the authoring of the docker-compose.yaml file as simple and seamless as the docker-compose.images.yaml override file solution. It could still be handy for some other usecases, so I wouldn't rule it out, but it's also possible to do this already by writing out an env file using bash scripts that run before the main installer and write out environment variables based on Porter template variables.

One bit of additional context here: I want to have docker-compose.yaml files that work standalone as well as as part of a bundle for ease of development. So for me, if Porter just injects environment variables, I still have to write my docker-compose.yaml in such a way that I use the porter variable when it's in a bundle but use plain image references otherwise.

There are two possible "sources of truth" for the image references: either the porter.yaml file or the docker-compose.yaml file. In the first case, testing the app during development would require getting Porter's image references to the docker-compose.yaml by means of environment variables.

However, it's much faster and more familiar for developers who already know docker-compose to develop and test using docker-compose up than porter install, so we treat the docker-compose.yaml file as the source of truth rather than the porter.yaml file. As such, when we go to bundle the app for installation, we need to copy the image references from the docker-compose.yaml file to the porter.yaml file for Porter to be able to construct a thick bundle with all of the images.

It would be really useful if mixins could add images to the images map at build time. Then the docker-compose-mixin could inspect the services in the docker-compose.yaml at build time and add them to the map for the author, eliminating the need to copy the image references into the porter.yaml manually or by means of external scripts. Other mixins could possibly do something similar.

[carolynvs]

What should the contents of docker-compose.images.yaml look like? If there's existing compose docs that would help someone implement this feature in the mixin, feel free to just link it here.

[sathish-kumar-narayanan]

Example docker-compose.yaml

services:
  frontend:
    image: awesome/webapp
    ports:
      - "443:8043"
    networks:
      - front-tier
      - back-tier
    configs:
      - httpd-config
    secrets:
      - server-certificate

Then, docker-compose.images.yaml would look like

services:
  frontend:
    image: <registry-url>/<image-name>:@sha

Effectively docker-compose.images.yaml will override the images from docker-compose.yaml when executing compose like this docker-compose -f docker-compose.yaml -f docker-compose.images.yaml up

https://docs.docker.com/compose/extends/

[carolynvs]

Thanks for the example that clearly explains the proposed solution!