Allow passing arbitrary config data to consul container

[afischer-opentext-com]

This PR adds the ability to pass arbitrary data to the consul container via a new environment variable CONSUL_EXTRA_CONFIG for configuration purposes.

Use case for this is that it allows to configure custom TLS (ca/cert/key_) file while avoiding the introduction of any persistence which needs to be maintained. This is e.g. beneficial when having consul running in AWS ECS as a side car as it allows to avoid the relatively complex overhead of an EFS volume while at the same time using the AWS secrets manager for storing the to be used secret data.

Command line example:

ls extra/
consulAgent.ca.pem  consulAgent.crt  consulAgent.key

EXTRA_CONFIG=$(tar --create --directory=extra/ . | base64)
LOCAL_CONFIG=$(cat <<-EndOfContent
{
  "bind_addr": "10.0.0.3",
  "ca_file": "/consul/extra/consulAgent.ca.pem",
  "cert_file": "/consul/extra/consulAgent.crt",
  "key_file": "/consul/extra/consulAgent.key",
  "leave_on_terminate": true,
  "rejoin_after_leave": true,
  "retry_join": [
      "10.0.0.2"
   ],
  "server": false
}
EndOfContent
)

docker run --net=host \
  -e "CONSUL_LOCAL_CONFIG=${LOCAL_CONFIG}" \
  -e "CONSUL_EXTRA_CONFIG=${EXTRA_CONFIG}" \
  consul

[hashicorp-cla]

All committers have signed the CLA.

[eculver]

Thanks for the contribution! I'm wondering if providing the config via the -config-file option directly in the CLI would work in a similar way? Something like this:

docker run
  -v "$LOCAL_CONFIG_PATH:/path/to/config" \
  consul -config-file /path/to/config/file.json

or even with an HCL code fragment like:

LOCAL_CONFIG_HCL=$(cat <<-EndOfContent
{
  bind_addr = "10.0.0.3"
  ...
}
EndOfContent
)
docker run --net=host consul -hcl "${LOCAL_CONFIG_HCL}"

The TLS CA and cert files could also be volume mounted and provided in a similar way w/ an argument to the agent.

In which cases, we wouldn't need this extra configuration argument?

[afischer-opentext-com]

Hi @eculver and thanks for looking into this.

And yes, agreed, there are alternatives. Given our use case with ECS the most obvious alternative would be to mount an EFS volume. We decided against it as it would cause the need to maintain and provision infrastructure only to pass the TLS files and in addition we would have needed logic which puts/updates the TLS files to the EFS volume. Our conclusion was that this is would be no more KISS.

Would the HCL code fragment support passing the TLS certificates, e.g. via base64 encoded string? To my understanding the certs can only be referenced but not added directly to the JSON/HCL document. This would be imho the very best approach as we would need to deal only with a single configuration fragment but not 4 of them. Unfortunately there's a lack of capability to add this feature on our end to the consul code base.

[eculver]

On the topic of KISS and adding config, I think this issue applies here: consul #10249. Specifically, if we had a way to pass secrets through to Consul via environment vars, we wouldn't have to make any changes here to the container definition. I think this would be the "simplest" solution.

However, in the absence of this functionality in Consul itself, I think we could still work around it without making changes to the container for the time being. To your second question:

Would the HCL code fragment support passing the TLS certificates, e.g. via base64 encoded string?

I don't think it needs to be embedded directly. These values could be still be referenced as volume mounts. This would be specific to your implementation but these values just need to be written out to files ahead of time and then referenced in the config via paths that are proper volume mounts. So, assuming the certs are on the system already at /consul/extra/consulAgent* (per your example), this path just needs to be provided to the container. Following my example:

docker run
  -v "/consul/extra:/consul/extra" \
  -v "$LOCAL_CONFIG_PATH:/path/to/config" \
  consul -config-file /path/to/config/file.json

Would this work for you? I don't quite see how the EFS volume is necessarily relevant here unless I'm missing something?

[afischer-opentext-com]

Our use case is ECS and the only way to mount volumes there is to go with EFS and this is what we attempt to avoid.

[eculver]

Our use case is ECS and the only way to mount volumes there is to go with EFS and this is what we attempt to avoid.

If you are using ECS, there are other ways to provide this path to Consul without having to mount an EFS volume. For example, these certs can be provided directly on the AMI where the ECS agent is running or populated at boot time with cloud-init.

While there is a little added complexity to link the paths into some sort of config reference, I'm still not clear on how that requires external infrastructure like EFS to accomplish it. Can you confirm whether the approach I mentioned above to populate the config file will work in your case? I'd like to avoid adding this extra option for the sake of maintaining a smaller API for the container. If this approach works, I think we can close this out.

[eculver]

Given I haven't heard anything in a few weeks on this, I'm going to assume that the last comment provided at least enough to chew on for now, so I'm going to go ahead and close this out. If anything changes or I was missing something, please feel free to re-open and we can discuss more.