Support configuring set of certificates in `/etc/ssl/certs`

[topherbullock]

Description

On the Concourse team we've had a long-running issue (concourse/concourse#1027) around adding certificates to resource containers. We need this to support custom docker registries, proxy servers, and various other super-enterprisey-custom-cert-requirin' use cases that are better served by a more general approach than special-casing each resource container's image / creation process.

After an approach of automatically bind-mounting certificates into all containers, we're wondering whether this would be better served as a feature of Garden itself.

A Modest Proposal

This could work in a similar to the way to how kawasaki writes / modifies /etc/hosts and /etc/resolv.conf based on network config passed to the Garden server. ServerCommand could have a separate namespace for Certificates, and allow specifying a set of cert files from the host to propagate over to created containers.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/146399843

The labels on this github issue will be updated when the story is started.

[julz]

the main problem we keep running in to when we try to find a way to do this within garden is there isn't really a standard for where certificates should land in a container or how they should be installed so without garden guessing what OS the container is running (which would be pretty bad) it's kind of hard to generically do this. Hence we've generally ended up punting this to higher levels.

Totally spit-balling, if only there was some sort of pipeline automation tool that could process a set of resource images and add certificates to them.. I hear there's this thing called concourse that's pretty great at that sort of thing..

[DRuggeri]

In practice /etc/ssl/certs is the most common location. For the resources concourse controls, it's easy to just use /etc/ssl/certs (because concourse knows exactly what OS is inside the container). For the OSs that wouldn't use /etc/ssl/certs, having something available in the container is better than having to create intermediate containers just to layer certs on it. If the OS doesn't use /etc/ssl/certs, most programs offer a way to choose a different path or CA file.

In any event, the proposal points out...

allow specifying a set of cert files from the host to propagate over to created containers.

which means guardian need not be responsible for determining where to propagate the certs nor even the process of installing them - it just needs to expose them. I read this to mean that the invoking process should have all that stuff set up.

[julz]

If it's just a case of installing the certs then this seems like you can just do it with bind mounts?

[julz]

sorry, I see you already mentioned that in the initial comment.. I guess my question is what's the downside of bind mounts here? With /etc/hosts and /etc/resolv.conf it's hard for the client to specify them because they don't know what IP we're going to give them, with the certificates it seems like the client knows better than garden which certificates to put at which location (because it knows which OS image it's going to use, and it knows which containers should get what certs). We could have some sort of set of default bind mounts that we put in every container but it doesn't seem super-hard for a client to do that itself?

[mariash]

I think managing OS configuration files should not be pushed up to higher levels. Concourse domain should not worry about what certificates, dns configuration, etc should be provided from the host. Concourse also does not necessarily know what OS is being used, it is some docker image name that we fetch and provide as rootFS to garden container. Also, since Diego was hitting the same issue it seems like it would be nice if that feature was baked in backend. Please let me know why you think this feature would not be appropriate in Garden if you think so.

I can see ca certificates configuration provided in your BOSH manifest by users as one of garden properties and garden will mount it as specified path. It does not have to be smart about reading OS from rootFS. User will be responsible for specifying certificates and destination and installation command themselves.

[mariash]

Just another idea how this might work. 1) User provides pem-encoded certs. Similar to how it is done in bosh: https://bosh.io/docs/trusted-certs.html. When using bosh user can reuse the same property value for certs. 2) User provides a path where these certs will be installed inside of container. 3) User provides a command to run to install those certs. Which is specific to OS. For example, for ubuntu it will be /usr/sbin/update-ca-certificates -f or for centos /usr/bin/update-ca-trust. Or if user wants to make that generic enough across all of their images they can just specify path to some execution script that will be baked in each image and can run corresponding command. 4) when container is spun up certs will saved in destination directory and the provided installation command will be run.

[julz]

Im going to close this for now since it's horribly old now. Although the use case makes total sense I think garden would need to start knowing/assuming too much about the rootfs for it really to happen inside garden. The algorithm above @mariash suggests seems entirely reasonable, but for a layer higher than garden I think (all the primitives needed -- bind mounts and ability to execute multiple commands before the main container process starts -- should be there in garden for a higher level system to do this).