ssm-secure dynamic reference resource type coverage

[candrews]

AWS::ElasticBeanstalk::Environment-Properties-OptionSettings[namespace==`aws:elasticbeanstalk:application:environment`].Value (environment variables values) should support Dynamic References to AWS Systems Manager Parameter Store Secure Strings.

Currently, Dynamic References to AWS Systems Manager Parameter Store Secure Strings are only supported in a limited set of places. It would be nice if they were supported in Beanstalk environment variable values (which are specified in CloudFormation at AWS::ElasticBeanstalk::Environment-Properties-OptionSettings[namespace==`aws:elasticbeanstalk:application:environment`].Value).

This should would allow the Beanstalk application to see an environment variable named ` with valuesupersecret` when defined by this CloudFormation template fragment:

---
AWSTemplateFormatVersion: '2010-09-09'
Resoures:
  BeanstalkEnvironment:
    Type: AWS::ElasticBeanstalk::Environment
    Properties:
      OptionSettings:
        -
          Namespace: "aws:elasticbeanstalk:application:environment"
          OptionName: SPRING_DATASOURCE_PASSWORD
          Value: !Sub "{{resolve:ssm-secure-env:/my/parameter:42}}"

6. Category (required) - Will help with tagging and be easier to find by other users to +1

Use the categories as displayed in the AWS Management Console (simplified):

1. Compute (Elastic Beanstalk)

[candrews]

It's possible to hack something like this using an ebextension today, but it really should be easier and supported directly in AWS.

[eballetbaz]

For information, this feature is already partially implemented into Beanstalk. It is working with non-secure parameters which specify the version:

i.e.: {{resolve:ssm:DB_PASSWORD:1}}

I tested with platform : Tomcat 8.5 with Java 8 running on 64bit Amazon Linux/3.3.0

Other options are not working (but pattern is recognized)

Secure reference, i.e. {{resolve:ssm-secure:DB_PASSWORD:1}} shows error:

Service:AmazonCloudFormation, Message:SSM Secure reference is not supported in: [AWS::CloudFormation::WaitConditionHandle/Metadata/AWS::ElasticBeanstalk::Ext/Parameters/EnvironmentVariables,AWS::AutoScaling::AutoScalingGroup/Metadata/AWS::ElasticBeanstalk::Ext/_ContainerConfigFileContent/optionsettings/aws:elasticbeanstalk:application:environment]

References without version, i.e. {{resolve:ssm-secure:DB_PASSWORD}} shows error:

Service:AmazonCloudFormation, Message:Incorrect format is used in the following SSM reference: [{{resolve:ssm-secure:DB_PASSWORD}}]

[paulmwatson]

Thanks @candrews but the formatting in your linked article has lost line breaks and indents. Would you mind pasting it again?

[candrews]

Thanks @candrews but the formatting in your linked article has lost line breaks and indents. Would you mind pasting it again?

Woops! I've fixed the article's formatting.

[amsinha2]

Is there any new on whether this will work?

[alex-all3dp]

I would love for something like this to be possible as well! It would be great if a .ebextensions config file could look like

option_settings:
  aws:elasticbeanstalk:application:environment:
    DB_USER: '{{resolve:secretsmanager:secretId:SecretString:DB_USER}}'
    DB_PWD: '{{resolve:secretsmanager:secretId:SecretString:DB_PWD}}'

[PatMyron]

generalizing similar

secretsmanager dynamic references are not limited to a hand maintained approved property type list. I see no reason why ssm-securedynamic references should be different

(opened an internal CR with relevant changes if anyone's looking to revive it)

[Juberstine]

@PatMyron What you said is exactly what we are looking for. This would be extremely helpful.

[shorif2000]

why do you need to do :1 in {{resolve:ssm:DB_PASSWORD:1}}

[pmoleri]

why do you need to do :1 in {{resolve:ssm:DB_PASSWORD:1}}

@shorif2000 It's the version. As per cloud formation documentation it's mandatory to indicate the version (at least for now).

I can confirm that as of this date:

• {{resolve:ssm:<name>:<version>}} works
• {{resolve:ssm-secure:<name>:<version>}} fails with:

  Service:AmazonCloudFormation, Message: SSM Secure reference is not supported in: [AWS::CloudFormation::WaitConditionHandle/Metadata/AWS::ElasticBeanstalk::Ext/Parameters/EnvironmentVariables,AWS::AutoScaling::AutoScalingGroup/Metadata/AWS::ElasticBeanstalk::Ext/_ContainerConfigFileContent/optionsettings/aws:elasticbeanstalk:application:environment]

I tested both as Environment Variables set directly in Elastic Beanstalk Console UI.

[mtHuberty]

Would really like to see this feature. It's something that's easily doable in a task definition in ECS, but isn't a well-solved problem yet in EBS. Using the CLI in ebextensions or in-app SDK calls as workarounds doesn't really fit the ease-of-use value prop that EBS seems to be going for.

[dnalbach]

Would really like to see this feature. It's something that's easily doable in a task definition in ECS, but isn't a well-solved problem yet in EBS. Using the CLI in ebextensions or in-app SDK calls as workarounds doesn't really fit the ease-of-use value prop that EBS seems to be going for.

I agree with this 100%. Elastic Beanstalk has no purpose other than to encapsulate other AWS services in an easy to use wrapper. Yet it does not support the the other AWS services that make Elastic Beanstalk usable securely. It's important for some teams to be able to permission Beanstalk and production credentials separately in IAM. That way people who shouldn't have access to those credentials don't, but can still make minor changes and do deploys in Beanstalk.

[cinnion]

One other thing I would like to see is to have documentation (or a video) go through the steps of setting up the variables for multiple environments. For example, we have a Laravel microservice we want to deploy in EB, with versions (environments) for testing and production. But what I have pieced together so far just seems to indicate that we would have to make changes to move between the two environments.

[peteawood]

Just ran into this yesterday. I can use secrets manager instead but as @PatMyron said, why should ssm-secure be limited? Would love to see this implemented :)

[webysther]

4 years ago I have implemented a datalake using an workaround and when migration to cdk v2 seems to be the same limitation, sad.

[atnjqt]

I guess folks have been struggling with this limitation for a while, as I just ran into this on Beanstalk. Trying to pass some third-party app unmanaged secret to the application environment variables... I'll probably just end up using a predeploy hook to export the variables to a chmod' 600 config.ini file that my flask app can readily read from

[andersb]

It's quite frustrating that this haven't been resolved yet.

In order to use either ssm-secure or secretsmanager, one needs to create some custom predeploy scripts to fetch secure string. This feels like it could be a bit of a hazel to maintain as there are new Linux profiles popping up every one in a while. On the other hand, using simple ssm works out of the box. So the question is, how terrible is it to place sensitive data (e g database password) in a normal string?

[atnjqt]

@andersb I have since realized the injection of secrets in an application deployment pipeline is a ubiquitously tricky thing (hence the popularity of stuff like HashiCorp Vault) -- of course, yes it's terrible and don't include secrets in plaintext!

Instead you can leverage platform pre-deploy hooks to accomplish this task. For reference see the docs here specific to AWS Elastic Beanstalk running Linux: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/platforms-linux-extend.html

Setup a bash script in .platform/common/get_param_store_values.sh with something of the following:

• This is specific to using Python configparser -- see ref here: https://docs.python.org/3/library/configparser.html
• I'm also parsing the AWS cli output using jq which is an invaluable tool -- see ref here: https://github.com/bobbyiliev/introduction-to-bash-scripting/blob/main/ebook/en/content/018-working-with-json-in-bash-using-jq.md

#!/bin/bash

set -e
USERNAME=$(aws ssm get-parameters --name "username-value" --output json | jq '.Parameters[]' | jq '.Value')
SECRET=$(aws ssm get-parameters --name "secret-value" --output json --with-decryption | jq '.Parameters[]' | jq '.Value')

set +x
# output your ssm values into a config file
echo "[SSM-CONFIGURATIONS]" > config_prod.ini
echo "USERNAME_VALUE = $USERNAME" >> config_prod.ini
echo "SECRET_VALUE = $SECRET" >> config_prod.ini

# restrict file permissions given these are secrets
sudo chmod 600 config_prod.ini
sudo chown webapp:webapp config_prod.ini
set -x

You would then call this script in a predeploy hook .platform/confighooks/prebuild/get_param_store_values.sh with the following:

../../common/get_param_store_values.sh

At the application layer (again assuming it's Python Linux on AWS EB) you can read that config_prod.ini value. You could even use Boto3 in python at the app layer to read the secrets but I thought it made sense to accomplish in deployment. Hope this is helpful!

[andersb]

Thanks @atnjqt for taking the time.

First side note, I'm using EB Tomcat and the application is based on Spring Boot. I rather not modify loading properties in the code itself so all environment properties should be set before launching Tomcat.

There are plenty of warnings about not storing sensitive data in regular Strings which sounds sane. However, what you are describing is a script that more or less stores all the secure strings in plain text in config_prod.ini, which you then load into your python app.

How is that in any way more secure than using resolve:ssm? I kind of assume that resolve:ssm will be using some kind of SSL-connection, and the result will very much likely land in the /var/log/messages in plain text, just like config_prod.ini

[atnjqt]

Thanks for the reply, yes my idea was largely to not commit the secret values to version control but in that sense yeah you're totally right I am writing plaintext in a file, modified with 600 permissions to lock for the web proxy owner... It's not perfect but definitely better than committing to git!

It's something I want to improve on 😅... maybe doing things at the application layer to pull in as an environment variable is preferable since it'll never get written to a file.

I did this referencing the Twelve Factor app rule III. Config as described here: https://12factor.net/config -- really I should get those secrets out of a config file and into an env var that the Beanstalk EC2 can access. I just wasn't at the time immediately sure how to accomplish that in a predeploy hook, though really maybe it should be application layer w/ Boto3 now that I'm thinking about it!

"Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not... A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials... The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard."

[andersb]

To be clear, I never mentioned committing sensitive data to git. That is the last thing I would opt for.

What I'm comparing here is having a config-file in .ebextensions which will contain something like the following

option_settings:
  aws:elasticbeanstalk:application:environment:
    my_property: '{{resolve:ssm:my_property:1}}'     << This is not super secure but maybe secure enough

with these workarounds for having a predeploy-hook-script that will get the secure strings and store them in plain text for Tomcat (or a python app) to pick up.

But to get back to the core issue, why can't AWS just implement support for resolve:ssm-secure or even resolve:secretsmanager from beanstalk?!!