search

arabold / serverless-export-env

Serverless plugin to export environment variables into a .env file
MIT License
103 stars 34 forks source link

export-env with sls offline not working because it's reaching the real cloudformation #24

Closed marooned071 closed 3 years ago

marooned071 commented 3 years ago

Hey,

Thank you for your contribution to the open-source community.

I would like to use that plugin in my test set up. It seems to fit perfectly for my use case. However, I found that export-env plugin is not working with serverless offline plugin. The problem is that during collecting environment variables, the plugin is trying to connect to the real AWS infrastructure to list variables and get stack output. Of course, those two operations will fail as nothing was deployed anywhere outside the local machine.

Error details:

  Serverless Error ---------------------------------------

  User: <user arn> is not authorized to perform: cloudformation:ListExports

and

  Serverless Error ---------------------------------------

  Stack with id <offline stack id> does not exist

If I'm using serverless-offline, there is no need for fetching anything from the real AWS. That is correct - right?

I did a bit of investigation. The problem is with collectResourcesOutputs function used in collectEnvVars and resolveEnvVars. They both are executed regardless of isOfflineHooked value and the that is causing errors.

arabold commented 3 years ago

Thanks, @marooned071 . I have merged your PR into v1.4.1 which is available via npm now.

coyoteecd commented 3 years ago

This PR is breaking our use of serverless-offline in combination with the serverless-export-env plugin. Connecting to the deployed stacks was/is useful in case your environment variables are hooked up with ImportValue calls, as described in #21. We use those to dynamically read outputs of other stacks, e.g. table names of Dynamo tables deployed by the db stack. The plugin would connect and get those values automatically when running serverless-offline, which was nice. Now, those env variables are all [object Object], making the local debugging unusable.

If I'm using serverless-offline, there is no need for fetching anything from the real AWS. That is correct - right?

Not for everybody, we use it to debug lambdas locally, while connecting to resources in AWS (databases/S3 buckets). The stack is there, we just run a "local copy" in debug mode. Much less hassle than figuring out how to set up a local copy of every AWS service your lambdas interact with.

I suggest reverting this PR and re-merging with a config option. For now I went back to 1.4.0.

@arabold, @marooned071 thoughts?

arabold commented 3 years ago

I was going through this again and I have to agree with @coyoteecd. I think this PR needs to be rolled back and reworked. Initially, I obviously didn't put much thought in, but it seems the new behavior based on whether offline is used isn't fully predictable for the user. There are legitimate use cases in which we want to update the environment file regardless of whether running using offline or not.

I'd be happy to bring this back but we would need to introduce an optional parameter instead. Maybe something even more generic such as "--ignoreErrors" that simply doesn't throw an error if the resources aren't accessible would be a suitable workaround?

arabold commented 3 years ago

Rolled back the PR. Update without the changes is in v1.4.2. I'm really sorry for the delay 😞

arabold commented 3 years ago

I'm currently working on a new v2.0.0. I've dropped support for some workarounds and introduced support for Fn::GetAtt and most other commonly used functions. This should make the use of the plugin much more straight forward. You can also disable offline support or skip creating a .env file depending on your personal use cases.

The version is not pushed to npm yet but a branch exits: https://github.com/arabold/serverless-export-env/tree/v2.0.0. I'm happy about any feedback before the official release.

arabold commented 3 years ago

To simplify testing the new version, I've published 2.0.0-alpha.0 as alpha release to npm:

npm install serverless-export-env@2.0.0-alpha.0 --save-dev

I'm closing this task. Please open a new one (or comment on this one) if your problems are still not solved.

joeyparis commented 3 years ago

I'm not sure I understand the proper way to handle this situation now with the current version (2.0.0). I am using serverless-offline and serverless-export-env and using Fn:GetAtt with the export-env.getAttMap field. I am getting the Stack with id <offline stack id> does not exist error, so I'm 99% sure it's related. Any help would be greatly appreciated!

arabold commented 3 years ago

In order to run export-env you will usually need access to the CloudFormation stack in AWS, as it has to pull the necessary information.

Specifically for !GetAtt there's a functionality to define the desired environment variable value in a mapping table using the getAttMap configuration setting in serverless.yaml: Custom Resource Resolution. This is not ideal (as you basically hard-code the value in the serverless.yml), but unfortunately AWS makes it extremely hard to (re-)implement the !GetAtt locally - it isn't fully supported yet.