[proposal] Add support for locally overriding env vars

[jamesls]

I'd like to override env vars in my config.json file only for local development purposes. I don't want to check in these values to source code and I don't want to modify the existing config.json.

As a concrete example, let's say I have a redis cluster in elasticache. Normally this is in a VPC, and I don't want it publicly accessible. I can configure my lambda functions in chalice to be in this VPC and deploy this code just fine.

However, for local development I'd like to point my chalice app at a dev instance of redis, perhaps running locally on my dev laptop. Right now I'm modifying my .chalice/config.json to override the real redis host/port with localhost, but I have to remember not to commit those changes.

Proposal

I'd suggest adding support for a .chalice/config.local.json. This file is deep merged in with the existing .chalice/config.json. We'd also add it to the generated .gitignore file as part of chalice new-project. So if I had:

$ cat .chalice/config.json
{
  "stages": {
    "dev": {
      "api_gateway_stage": "api",
      "environment_variables": {
        "REDIS_HOST": "https://elasticache....amazonaws.com/",
    "REDIS_PORT": "1234"
      },
  ...
}

I could create a local config file like this:

$ cat .chalice/config.local.json
{
  "stages": {
    "dev": {
      "api_gateway_stage": "api",
      "environment_variables": {
        "REDIS_HOST": "localhost",
    "REDIS_PORT": "5678"
      },
  ...
}

and my local copy of my chalice app would use my dev instance of redis.

What do you all think? How are others solving this problem?

[jamesls]

Going through older proposals, I was reminded of a similar proposal: https://github.com/aws/chalice/issues/608#issuecomment-345391098

That proposal is more general at the cost of being more complex for people to use. However, if we did end up implementing "app parameters", I'm not sure if we'd also want the .config.local.json. You could potentially just have a local-params.json that would let you override these env vars with your local values as needed. It's worth thinking about before going forward with either proposal.

[sciurus]

I'm also looking for a solution for this, as I do not want to check in actual values for my environment variables. #608 seems to fit my needs better.

[onlyanegg]

I think, at the least, chalice local should not default to using the 'dev' stage; it should default to using the 'local' stage. I think we can agree that environment variables set for a dev environment in AWS should not necessarily be the same as those for a local environment. Currently I have to do chalice local --stage local to emulate this, which is too verbose IMHO. This would allow users to specify local environment variables within the config.json or ignore it and set local environment variables using some other method (eg. dotenv).

[dmulter]

The solution I use works for secure production/stage/dev env vars. I don't do local deploys, but I'm sure this technique can be extended. I keep all secure vars in SSM. During deploy I retrieve the SSM params and then add them to the deployed Lambda using get_function_configuration() and update_function_configuration(). I avoid using SSM from my Lambda functions since it's easy run into SSM rate limits that can't be raised.

There is a small delay when the env vars are not available to the Lambda function, but I handle that in my function. This also makes it easy to use KMS to encrypt secret vars, then I use a decrypt in my function. This whole dance could be avoided if these steps were done by Chalice. IMHO, this is a giant whole in real Chalice deploys.

[gitblit]

There are multiple related ideas presented in this issue which seem really logical. Two of them I wish were already part of Chalice.

• support for .chalice/config.{stage}.json
• simplifying chalice local --stage local (maybe a config.json setting to specify a default stage for chalice local)

I'd like to be able to use chalice local completely offline from AWS for API development without version controlling my setup while making it easy to manage or share config.

I see there is an attempt to use Github Projects for roadmapping. Is this issue actively under consideration for a next release?

[nileshprasad137]

+1 to this feature request. Would really like this feature in chalice.

[nickovs]

I would also like to see this happen, to solve the use case described in #1334. If the local config overlay file is added to .gitignore then you can put your secrets there and avoid them being distributed, while maintaining version control on all of the plumbing.

[josiahcoad]

Bump allowing something like. chalice deploy -e "my_var=${GITLAB_VAR}" I am using github actions for CICD and this would allow me to inject env vars stored in github secrets.