amplify env checkout causes local changes to lambda layer

jamescrowley commented 3 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

6.0.0

What operating system are you using?

Mac

Amplify Categories

function

Amplify Commands

env

Describe the bug

Whenever I switch between environments using amplify env checkout X that have a lambda layer, there are changes made to the awscloudformation-template.json and parameters.json for the layer.

Expected behavior

amplify env checkout X should not result in any changes to files that are git committed.

Reproduction steps

I'm not clear yet whether this is expected or not - happy to try to reproduce further if useful.

I have one lambda function, with one lambda layer.

backend-config.json:

 "function": {
    "LambdaWithLayer8ba9f268": {
      "service": "Lambda",
      "providerPlugin": "awscloudformation",
      "build": true,
      "dependsOn": [
        {
          "category": "function",
          "resourceName": "LambdaLayer",
          "attributes": [
            "Arn"
          ]
        }
      ]
    },
    "LambdaLayer": {
      "providerPlugin": "awscloudformation",
      "service": "LambdaLayer",
      "build": true
    }
  },

function parameters.json:

{
  "lambdaLayers": [
    {
      "type": "ProjectLayer",
      "resourceName": "LambdaLayer",
      "env": "dev",
      "version": "Always choose latest version",
      "isLatestVersionSelected": true
    }
  ]
}

Log output

``` # Put your logs below this line ```

attilah commented 3 years ago

@jamescrowley is your amplify env checkout involves git branch switching as well? dev env == dev branch, prod env == prod branch? or you are using 1 git branch for all of the envs?

If you could share the command sequence you are using would help better understand your concern and answer the question if this is by design or a known side effect or something we could fix.

jamescrowley commented 3 years ago

That's the intention ultimately but currently all point to the same commit and I get the same behaviour when staying on the same branch. So for clarity, I see the behaviour simply doing

amplify env checkout dev amplify env checkout prod

with no other commands being run

nathanagez commented 3 years ago

Hi, I noticed the same behavior on my side, it's annoying since I need to push new lambda layer versions each time, even if I did not updated them.

It only happens when changing environment.

arbieo commented 3 years ago

We have encountered the same issue and the function parameters.json changes make us fairly distressed. Using the example above:

{
  "lambdaLayers": [
    {
      "type": "ProjectLayer",
      "resourceName": "LambdaLayer",
      "env": "dev",
      "version": "Always choose latest version",
      "isLatestVersionSelected": true
    }
  ]
}

Specifically:

"env": "dev",

This file is supposed to environment agnostic, is my understanding. It feels wrong to commit this value and share it with my team/go to prod with it. Would my production functions be referencing layers in dev due to this configuration?

mattfysh commented 2 years ago

Just ran into this same issue, I'm using the AWS OpenTelemetry lambda layer for nodejs. Every time I checkout a new environment, it deletes the lambda layer from function-parameters.json

413n commented 2 years ago

+1. Same problem of @arbieo. Diffs between dev and prod branches for the lambda layer env, consequently brings to merging problems

maxipaz commented 2 years ago

Same problem of @arbieo here too. It is very annoying to have this modified all the time.

bobfix commented 2 years ago

We are seeing the same behavior as well. Another problem that this issue causes is that if your Cloud Formation stack has to be rolled back for any reason, it will fail due to this.

gspeicher commented 2 years ago

Obligatory +1 / me too comment for everything described above, but for us the layer cloudformation template itself is also different across environments due to all of the LayerVersionPermission resources generated (one for each layer version), each of which then also references the environment name directly, e.g.:

    "LambdaLayerPermissionPrivate2fbef436": {
      "Type": "AWS::Lambda::LayerVersionPermission",
      "Properties": {
        "Action": "lambda:GetLayerVersion",
        "LayerVersionArn": "arn:aws:lambda:us-east-1:XXXXXXXXXXXX:layer:exampleLayer-master:12",
        "Principal": {
          "Ref": "AWS::AccountId"
        }
      }
    },

Note that master is the environment here because this is a master backend checkout, but if I checkout our dev environment then all these LayerVersionPermissions are replaced with different ones. This makes it hard to track in separate master and dev branches of a git repository.

Also kind of amusing that AccountId is referenced in Principal after it is hardcoded in the ARN, but at least that is not causing us any tooling/process issues.

josefaidt commented 2 years ago

cc @edwardfoyle @swaminator fyi for multi-env redesign

dral3x commented 2 years ago

+1 on this issue. Using Amplify CLI 9.1.0.

brauliolledo commented 2 years ago

Any news on this? This issue is causing confusion on our multienv/multibranch development flow.

lincetto commented 2 years ago

+1 Same problem, no solutions found yet

dr-star commented 2 years ago

+1 Same problem, no solutions found yet

Stefan-WW commented 1 year ago

same problem here

kimmosaarinen commented 1 year ago

+1. Yet another annoying feature of Amplify. Using CLI 11.0.3.

danielvouch commented 1 year ago

+1 - Any update on this?

dan-hook commented 1 year ago

We have encountered the same issue and the function parameters.json changes make us fairly distressed. Using the example above:
{
  "lambdaLayers": [
    {
      "type": "ProjectLayer",
      "resourceName": "LambdaLayer",
      "env": "dev",
      "version": "Always choose latest version",
      "isLatestVersionSelected": true
    }
  ]
}
Specifically:

"env": "dev",

This file is supposed to environment agnostic, is my understanding. It feels wrong to commit this value and share it with my team/go to prod with it. Would my production functions be referencing layers in dev due to this configuration?

I replaced my functionparameters.json files with Jinja2 templates that fill in the environment name with a pre-push hook with all sorts of checks to make sure the template stays in sync with any changes to the functionparameters.json. It's an insane solution, and I can't bring myself to apply the same fix for the awscloudformation-template.json.