"chalice deploy" removes Lambda environment variables

[chrislawlor]

Rather than adding encrypted environment variables to my config.json file, I'd prefer to simply add them manually to Lambda (either via the web console or CLI). However, all manually added environment variables are currently being removed on every deploy. This is occurring in a project with no environment_variables section in the config file.

I would expect the following behaviour:

1. If there is no environment_variables section, all configured env vars are persisted.
2. If there is an environment_variables section, all non-conflicting variables should be merged. So if I have manually set FOO, and my environment_variables section contains BAR, the final configuration should contain both FOO and BAR.
3. For existing env vars, the environment_variables section should take precedence.

For context, support for Lambda environment variables was added here in PR #243 , and support for environment variables in the .chalice/config.json file was added in PR #273.

[JordonPhillips]

Can I ask what about the environment variable support isn't working out for you? Is there anything we can do to help out your use case?

In general, I would be supportive of having a way to merge changes that have been made to live. A good way to do that would be to have some sort of sync command that detects changes made remotely and then modifies your local configuration to match. That way your chalice project is still the source of truth. How does that sound?

[chrislawlor]

The thing I'm trying to avoid is storing environment variables in files that are checked in to version control. Perhaps there's some way to do this in some low friction way that doesn't involve setting up encryption services that I otherwise don't need, it just seems like a heavyweight solution to a simple problem. So, merging manually configured environment variables to the configuration file would succeed in persisting them across deploys, but it would also somewhat defeat the purpose of having them in the first place.

[chrislawlor]

I should also mention that, while I don't currently have a great understanding of what a deploy does behind the scenes, I'd be willing to work on a patch for implementing the functionality I described above, I'd like to get some validation on the idea, or otherwise get to some consensus on how things should work, before I start actually working on it.

[bitblit]

I've run into this as well - My workaround has been to write my own script that runs after the chalice deploy and resets the EnvVars but thats a hack. My proposal : If chalice could do environmental expansion on variables before upload, i.e., assuming I have an environmental variable ABC (set to 123) on my machine, I could put this in the config:

"environment_variables": { "NEW_ABC": "${ABC}", ... }

and on the server side get

NEW_ABC = 123

That would work both locally and fit in easily with most CI solutions like CircleCI.

---

Incidentally, I don't think the original issue listed (other vars getting blown away) is actually a Chalice issue - its how lambda does variable setting... at least it matches how aws cli does it (which makes sense given CLI's BOTO underpinnings). To fix the merge problem I think Chalice would likely have to pull the current variables, update them with the new ones from the config, and then post them back - at least until lambda offers some form of "merge variables" endpoint

[syabro]

Vote up for it. KMS is nice but looks like an overengineering from my point of view. Also I want to change parameters by my manager who prefer to use web interface rather than run weird script from the console.

[sciurus]

The exact solution here initially (allow manual edits to the lambda to persist between chalice deploys) feels odd, but a solution that allows setting environment variables for the lambda without storing the values in config.json is definitely needed.

608 and #838 are related.

[tmcfarlane]

I thought Lambda code was independent of the environment variables. It would seem like chalice re-deploys the whole lambda every time. Why not add feature to only update code? chalice deploy --code-only If that fails because of lambda configuration then that's on us.

AWS supports this with update-function-code with these stipulations (taken from the page linked):

Updates the code for the specified Lambda function. This operation must only be used on an existing Lambda function and cannot be used to update the function configuration.

If you are using the versioning feature, note this API will always update the $LATEST version of your Lambda function. For information about the versioning feature, see AWS Lambda Function Versioning and Aliases .

If there is enough demand for this, I would love to take a whack at it.

[justin-ad]

Encountering this issue as well. I could use secrets manager to store and retrieve secrets, but I still need to put the access key id/secret access key somewhere to be able to connect to secrets manager via boto3 (the classic secrets storage problem). Currently the only way I'm able to accomplish this is by putting the access key id/secret access key in chalice/config.json environment_variables: {}, which is obviously sub optimal... I'm currently omitting this file from VCS to prevent leaking these secrets, but that's only a temporary solution.

[guillemmateos]

For sensitive data, secrets manager makes a lot of sense. If you assign permissions via IAM, the only thing you should have to reference on your function is the secret path (you could even use parameter store) which is not sensitive information. @jtwolgamott you shouldn't have to specify any confidential info in the files you commit to VCS if you assign the right IAM permissions to the execution role of your lambda; only the id of the secret is needed.