Encrypt Variables

[ktruckenmiller]

Hey! Awesome job by the way. I really love using this. There are a TON of things that this can give us that I don't have to build.

But I have a feature request (if it isn't already done)

With the cloudformation, can you specify encrypted? That way, when we use 'aws cloudformation deploy' it will go through, and encrypt with the KMS key you specify?

Something like this maybe

Environment:
  KmsKeyId: 1234-1234-1234-1234
  Variables:
    NON_SECRET: blah
    SECRET: !Encrypt this_gets_encrypted

Then we can just do the regular boto.kms.decrypt when we need to decrypt it in the lambda.

[ktruckenmiller]

Also, with the documentation that you have on this github, it would be awesome if you put:

Environment:
  Variables:
     ENV: boston

Instead of just 'Variables: '

I got confused : /

[vikrambhatt]

Thanks for the feedback and glad that you like SAM.

Cloudformation does let you specify a KMS key that lambda will use to encrypt and decrypt your environment variables.

With this setup, you will not be decrypting the variables inside your lambda function code - lambda will do it using roles you provided. More description here.

Cloudformation and SAM do not currently support encryption helpers that Lambda console provides. I think this is what you are looking for - Ability to encrypt selected variables, and decrypt them inside lambda function code. As of now the only way to achieve this (other than through console) is to use client side encryption - use encrypted cipher text in your variables and then decrypt them in the lambda function code.

[ktruckenmiller]

Yup. I think this would be great. That way on a aws cloudformation package it would encrypt on the clientside using boto3 / kms, and then would send the cloudformation template that way, just like it does with s3 objects when it packages.

[kpx-dev]

FYI I did this in 3 steps:

1) Encrypt the key using aws cli: aws kms encrypt --key-id KMS_KEY_ID --plaintext THE_SECRET_TOKEN

2) Include the encrypted token, and KmsKeyArn that you used in Step 1 inside my SAM yaml:

KmsKeyArn: arn:aws:kms:us-east-1:xxx:key/yyy
Environment:
        Variables:
          SECRET_TOKEN: '...524krvmX9JA='

3) Within the lambda, decrypt the ENV. Ex using Python

import boto3
import os
from base64 import b64decode

ENCRYPTED = os.environ['SECRET_TOKEN']
DECRYPTED = boto3.client('kms').decrypt(CiphertextBlob=b64decode(ENCRYPTED))['Plaintext'].decode('utf-8')

With this approach, I can check in the encrypted key inside git and I don't have to enter the key from UI Console manually.

[FLavalliere]

Just thinking out load.

Azure have the concept of the "Vault" as a Managed free service. The values created, can them be referenced from an ARM Template see https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-keyvault-parameter#reference-a-secret-with-static-id

Any option to do something similar with AWS ?

[jpompe-classy]

I solved this problem by creating a custom CFN resource that takes in a NoEcho parameter value and returns an encrypted blob to use in the lambda functions environment. The functions have appropriate decrypt/encrypt permissions via the key policy. See this fork: https://github.com/jpompe-classy/lambda-backed-cloud-formation-kms-encryption

[FLavalliere]

@jpompe-classy oh nice, agreed. Using the NoEcho parameter is the best solution here. Forgot about that.

Thanks.

[pinemind]

@vikrambhatt

Perhaps this isn't the right forum, but I'm working with this concept in my new role, and I'm having trouble understanding where the problem actually lies regarding the encryption of environment variables. To me, it seems perfectly capable right now with CloudFormation to encrypt and decrypt environment variables that are used in Lambda functions, simply by providing the KmsKey:

Cloudformation does let you specify a KMS key that lambda will use to encrypt and decrypt your environment variables ... With this setup, you will not be decrypting the variables inside your lambda function code - lambda will do it using roles you provided.

Maybe I am a newbie to the intricacies of security, but if Lambda does is using the roles provided along with the KmsKey to encrypt and decrypt, where is the security issue? I know you say that you "will not be decrypting the variables inside the Lambda function code" so where is the hole? Is it because they are decrypted at runtime via the role and KMS, and then transmitted back into the Lambda function as plain text? Whereas if we had helper code inserted (like in the console), the environment variables would always exist as Ciphertext until that line of helper code is executed during runtime, thus eliminating the possibility of someone intercepting the call from the Lambda function to KMS via the role to view the decrypted environment variable?

[sanathkr]

@pinemind You're right. The idea is you encrypt the variables before calling any AWS service. So there is no chance of exposure what-so-ever. Secrets like passwords, credit card information, etc are so sensitive that you shouldn't take any chance.

SAM isn't built to encrypt variables for you on the backend. You need to use a client-side tooling to do the encryption for you. The workflow that @kienpham2000 suggested is the best to follow.

Closing this Issue because SAM can never implement it in the backend.

[pacey]

Is there any documentation on the !Encrypt intrinsic function?

My understanding of the function is that it will encrypt a string and then decrypt it for you in the lambda execution environment. The obvious issue with that is that the value has to be in plain text in your CFN template, or template parameters. It would be awesome if there was a !Decrypt function, where you could put KMS encrypted value in the CFN template and get the decryption functionality in the lambda environment.

We currently use the method of calling KMS in our function. We've noticed that it's pretty slow, about 4 seconds to decrypt.

[kpx-dev]

@pacey I don't think !Encrypt function exist, it's just a suggested function for AWS to implement.

I recently learn about AWS SSM (System Store Manager), it's like Hashicorp Vault where you can store environment there and the good thing is CloudFormation can read from it. I learned about it from this blog post: https://aws.amazon.com/blogs/mt/integrating-aws-cloudformation-with-aws-systems-manager-parameter-store/

So I've been using it to store my Secret there instead of using the 3 steps I suggested above. So it's 2 steps now:

1) Update / Create the SSM key using AWS CLI:

aws ssm put-parameter --name "/dev/DB_PASS" --type "String" --value "StRONGPa$$%%@!!!"

2) In your sam template, import it like below. SSM supports string, list and secret, but looks like CloudFormation doesn't support secret field yet, so we have to store secret as string, still pretty secure if no one have access to your AWS SSM.

Parameters
  EnvDBPass:
    Type : 'AWS::SSM::Parameter::Value<String>'
    Default: /dev/DB_PASS

....

Resources:
  NetWorthFunction:
      Environment:
        Variables:
            SECRET_KEY: !Ref EnvDBPass

Full example: https://github.com/networth-app/networth/blob/040295e9dd9422ce67d7c9c872a4936d43ada4b2/api/template.yml

[hoang-innomize]

We wanted to know secured string for local and deployment purpose, it won't be used for runtime such as we wanted to configure VpcConfig but we don't want to use plaintext for security group id, subnet id. The configuration is per environment (different between test and prod, able to toggle VpcConfig). When we are working with serverless framework, we are able to implement using https://github.com/serverless/serverless-secrets-plugin.

Does anyone know any idea to implement using SAM?

[keetonian]

@hoang-innomizetech Any values used for VpcConfig and in other areas of SAM need to be in plain text, which means that these values would be visible in the CloudFormation console. You could use parameter overrides to pass in these values to the template from wherever they are stored so they are not stored locally.

[hoang-innomize]

@keetonian Yeah, we use parameter but we don't have any way to toggle it such as disable for non-production environments.

Globals:
  Function:
    Runtime: nodejs8.10
    Timeout: 600
    MemorySize: 1024
    VpcConfig:
      SecurityGroupIds: !Split [',', !Ref SecurityGroupIds]
      SubnetIds: !Split [',', !Ref PrivateSubnetIds]
    Environment:
      Variables:
        ENV: !Ref 'Environment'

How can we disable or ignore configure Vpc for Lambda if these parameters are Empty or not supplied?

[kay-is]

@vikrambhatt is there a way to encrypt the variables myself with the default KMS service key for Lambda?

The idea that Lambda automatically decrypts for me sounds nice, but I don't want to have my variables in cleartext in my VCS.