Support AWS SSM Parameter store for Environment Variables

[OverlordQ]

SSM Allows the creation of SecureString's which are variables protected by a KMS string which allows you to store sensitive information in a secure manner.[1]

This is the preferred method of storing environment or other data in plaintext as it stays encrypted in-flight.

[kylegibson]

More context:

https://medium.com/@tdi/ssm-parameter-store-for-keeping-secrets-in-a-structured-way-53a25d48166a

from the "Paths" section:

Parameters can be named either by a simple string or a path. When you use path — you introduce hierarchy on your parameters. This makes it easy to group parameters by stage, app or whatever structure you imagine. SSM allows you to get parameters by path.

Let’s say we have parameters:

• /myapp/production/DB_NAME
• /myapp/production/DB_PASSWORD
• /myapp/production/DB_USERNAME

In order to get all of them you would do:

aws ssm get-paramaters-by-path --with-decryption --path /myapp/production

This will produce you with a JSON array containing all of the parameters above.

[kylegibson]

So I got it working... without having to change anything in Zappa:

def populate_environ_from_ssm():
    import os
    ssm_param_path = os.getenv('AWS_SYSTEMS_MANAGER_PARAM_STORE_PATH')
    if not ssm_param_path:
        return
    import boto3
    client = boto3.client('ssm')
    response = client.get_parameters_by_path(Path=ssm_param_path, WithDecryption=True)
    for param in response['Parameters']:
        env_name = os.path.basename(param['Name'])
        os.environ[env_name] = param['Value']

populate_environ_from_ssm()

I'm calling this from my django settings module, before anything else.

Then update your zappa settings:

  aws_environment_variables:
    AWS_SYSTEMS_MANAGER_PARAM_STORE_PATH: /project-name/stage-name

(The path example above is just an example, but it's probably a good practice)

Next, the lambda execution role needs to have sufficient permissions to read from SSM parameter store.

It works.

Edit: If you dislike this solution, I'm very interested in understanding why.

[trickidicki]

Suggestion - perhaps throw an exception if you're running on Lambda and the SSM parameter isn't set:

    import os
    if os.environ.get('AWS_EXECUTION_ENV') is not None:
        ssm_param_path =os.environ['AWS_SYSTEMS_MANAGER_PARAM_STORE_PATH'] 
        ...

[kylegibson]

Good idea. Another change I've made is to skip envs that are already set.

[brylie]

I like the solution @kylegibson proposes in general, but have slight reservation about grabbing all parameters in a given namespace:

response = client.get_parameters_by_path(Path=ssm_param_path, WithDecryption=True)
    for param in response['Parameters']:
        ...

Perhaps this is a non-issue, but it might be good to be more explicit about which parameters to fetch. I.e. we usually specify a one-to-one mapping of parameters to Python variables, so we don't grab more than we need.

[ezegolub]

How would this deal with changing one of the secureStrings in KMS? You'd need to make some change to the code to trigger a reload on the lambda-side right? Great solution BTW.

[kylegibson]

How would this deal with changing one of the secureStrings in KMS? You'd need to make some change to the code to trigger a reload on the lambda-side right?

Every new lambda does a fresh load of the currently set parameters. I have not experienced any need to change the code to trigger a reload on the lambda-side.

However, when you edit a parameter, it creates a new version of that parameter (rather than changing the existing version). That's all fine, unless you are explicitly pinning the parameter version in your code. If you are doing that, then you will need to update the code to point to the new version of the parameter.