search

aws-amplify / amplify-cli

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development.
Apache License 2.0
2.81k stars 821 forks source link

Function parameters.json file being ignored in new deployment. #13655

Closed duckbytes closed 6 months ago

duckbytes commented 7 months ago

How did you install the Amplify CLI?

No response

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

12.10.1

What operating system are you using?

Linux

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made.

Describe the bug

I have several lambda functions with parameters provided in parameters.json. Parameters.json is in the root of the function, e.g. amplify/backend/function/plateletAddNewTenant/parameters.json and contains data like:

{
    "plateletWelcomeEmail": "info@platelet.app",
    "plateletDomainName": "app.platelet.app"
}

In the past this is how I've provided environment variables to functions. Existing deployments and new deployments have worked with this setup.

Recently I updated my lambda functions to node 18 and also updates the build settings to use the latest image and CLI version:

image

However I receive this output on the Amplify console.

2024-03-17T20:21:45.455Z [WARNING]: āœ– There was an error initializing your environment.
2024-03-17T20:21:45.455Z [INFO]: šŸ›‘ This environment is missing some parameter values.
2024-03-17T20:21:45.456Z [INFO]: [plateletDomainName,plateletWelcomeEmail,plateletDomainName,plateletWelcomeEmail,plateletDomainName,plateletWelcomeEmail,plateletSendFromEmailAddress,plateletSendToEmailAddress] do not have values.
                                 Resolution: Run 'amplify push' interactively to specify values.
                                 Alternatively, manually add values in SSM ParameterStore for the following parameter names:
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAddNewTenant_plateletDomainName
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAddNewTenant_plateletWelcomeEmail
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminAddNewUser_plateletDomainName
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminAddNewUser_plateletWelcomeEmail
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminResetUserPassword_plateletDomainName
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminResetUserPassword_plateletWelcomeEmail
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletSendUserFeedback_plateletSendFromEmailAddress
                                 /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletSendUserFeedback_plateletSendToEmailAddress
                                 Learn more at: https://docs.amplify.aws/cli/reference/ssm-parameter-store/#manually-creating-parameters
2024-03-17T20:21:45.493Z [INFO]: 
2024-03-17T20:21:45.493Z [INFO]: Session Identifier: 787d8900-f996-4d45-8bb8-cbede870d338
2024-03-17T20:21:45.516Z [ERROR]: !!! Build failed
2024-03-17T20:21:45.516Z [ERROR]: !!! Error: Command failed with exit code 1
2024-03-17T20:21:45.517Z [INFO]: # Starting environment caching...
2024-03-17T20:21:45.517Z [INFO]: # Environment caching completed
Terminating logging...

I'm triggering these builds with pushes to github.

This is my amplify.yml:

version: 1
env:
  variables:
    REACT_APP_OFFLINE_ONLY: false
    REACT_APP_POPULATE_FAKE_DATA: false
    REACT_APP_DEMO_MODE: false
    REACT_APP_RESIZE_BUCKET_NAME: platelet-resize-bucket
    REACT_APP_DEMO_PROFILE_PICTURES_BUCKET_NAME: platelet-demo-profilepics
    REACT_APP_DEMO_PROFILE_PICTURES_BUCKET_REGION: eu-west-1
backend:
  phases:
    build:
      commands:
        - '# Execute Amplify CLI with the helper script'
        - npm install -g prebuild-install --unsafe-perm=true
        - amplifyPush --simple
frontend:
  phases:
    preBuild:
      commands:
        - npm install
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: build
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*
test:
  artifacts:
    baseDirectory: cypress
    configFilePath: '**/mochawesome.json'
    files:
      - '**/*.png'
      - '**/*.mp4'
  phases:
    artifacts: # arbitrary values (other than baseDirectory) bc amplify test phase will fail without them
      baseDirectory: ./
      configFilePath: '**/mochawesome.json'
      files:
        - '**/*.png'
        - '**/*.mp4'
    preTest:
      commands:
        - npm ci
    test:
      commands:
        - echo "no tests"

I am not committing team-provier-info.json and would rather these values are taken from the parameters.json file if that's possible.

Expected behavior

I would expect building to use the values in parameters.json for functions instead of saying they are missing.

Reproduction steps

  1. Add a function to your project
  2. Include parameters when creating the function
  3. Create parameters.json in the function directory
  4. Place values in that file.
  5. Set up Branch autodetections in Repository settings (I have it set to production/**)
  6. Push to a new github branch production/something

Project Identifier

No response

Log output

``` # Put your logs below this line 2024-03-17T20:20:05.760Z [INFO]: # Cloning repository: git@github.com:platelet-app/platelet.git 2024-03-17T20:20:08.673Z [INFO]: 2024-03-17T20:20:08.673Z [INFO]: Cloning into 'platelet'... 2024-03-17T20:20:08.674Z [INFO]: # Switching to commit: 6ec2d08f5a397fdc631ce3b1210a9d65e819a9b4 2024-03-17T20:20:08.725Z [INFO]: Note: switching to '6ec2d08f5a397fdc631ce3b1210a9d65e819a9b4'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 6ec2d08f print amplify version 2024-03-17T20:20:08.779Z [INFO]: Successfully cleaned up Git credentials 2024-03-17T20:20:08.779Z [INFO]: # Checking for Git submodules at: /codebuild/output/src1790987902/src/platelet/.gitmodules 2024-03-17T20:20:08.790Z [INFO]: # Retrieving environment cache... 2024-03-17T20:20:08.831Z [WARNING]: ! Unable to write cache: {"code":"ERR_BAD_REQUEST","message":"Request failed with status code 404"})} 2024-03-17T20:20:08.832Z [INFO]: ---- Setting Up SSM Secrets ---- 2024-03-17T20:20:08.832Z [INFO]: SSM params {"Path":"/amplify/dlkp8fs0a5r7s/production/another.platelet.app/","WithDecryption":true} 2024-03-17T20:20:11.434Z [INFO]: # Patching NPM package '@aws-amplify/cli' from 12.8.2 to 12.10.1... 2024-03-17T20:20:38.935Z [INFO]: # Installed '@aws-amplify/cli@12.10.1' 2024-03-17T20:20:39.035Z [INFO]: # Done patching NPM package '@aws-amplify/cli' to version 12.10.1 2024-03-17T20:20:39.052Z [INFO]: # Retrieving cache... 2024-03-17T20:20:39.091Z [INFO]: # Retrieved cache 2024-03-17T20:20:39.282Z [INFO]: BackendEnvironment name production/another.platelet.app for app dlkp8fs0a5r7s is invalid 2024-03-17T20:20:39.358Z [INFO]: Random environment name saloma generated 2024-03-17T20:20:43.789Z [INFO]: ## Processing Environment Variables # Set Environment Variable: REACT_APP_OFFLINE_ONLY # Set Environment Variable: REACT_APP_POPULATE_FAKE_DATA 2024-03-17T20:20:43.798Z [INFO]: # Set Environment Variable: REACT_APP_DEMO_MODE # Set Environment Variable: REACT_APP_RESIZE_BUCKET_NAME # Set Environment Variable: REACT_APP_DEMO_PROFILE_PICTURES_BUCKET_NAME # Set Environment Variable: REACT_APP_DEMO_PROFILE_PICTURES_BUCKET_REGION ## Starting Backend Build # Starting phase: build # Executing command: npm install -g prebuild-install --unsafe-perm=true 2024-03-17T20:20:45.365Z [INFO]: added 37 packages in 1s 2024-03-17T20:20:45.366Z [INFO]: 9 packages are looking for funding run `npm fund` for details 2024-03-17T20:20:45.371Z [INFO]: # Executing command: amplify --version 2024-03-17T20:20:46.617Z [INFO]: 12.10.1 2024-03-17T20:20:48.588Z [INFO]: Note: It is recommended to run this command from the root of your app directory 2024-03-17T20:20:48.828Z [INFO]: Amplify AppID found: dlkp8fs0a5r7s. Amplify App name is: platelet-production 2024-03-17T20:20:48.872Z [INFO]: Adding backend environment saloma to AWS Amplify app: dlkp8fs0a5r7s 2024-03-17T20:20:54.205Z [INFO]: 2024-03-17T20:20:54.212Z [INFO]: CREATE_IN_PROGRESS amplify-platelet-saloma-202048 AWS::CloudFormation::Stack Sun Mar 17 2024 20:20:49 GMT+0000 (Coordinated Universal Time) User Initiated CREATE_IN_PROGRESS AuthRole AWS::IAM::Role Sun Mar 17 2024 20:20:51 GMT+0000 (Coordinated Universal Time) CREATE_IN_PROGRESS DeploymentBucket AWS::S3::Bucket Sun Mar 17 2024 20:20:51 GMT+0000 (Coordinated Universal Time) CREATE_IN_PROGRESS UnauthRole AWS::IAM::Role Sun Mar 17 2024 20:20:51 GMT+0000 (Coordinated Universal Time) CREATE_IN_PROGRESS UnauthRole AWS::IAM::Role Sun Mar 17 2024 20:20:52 GMT+0000 (Coordinated Universal Time) Resource creation Initiated CREATE_IN_PROGRESS AuthRole AWS::IAM::Role Sun Mar 17 2024 20:20:52 GMT+0000 (Coordinated Universal Time) Resource creation Initiated CREATE_IN_PROGRESS DeploymentBucket AWS::S3::Bucket Sun Mar 17 2024 20:20:52 GMT+0000 (Coordinated Universal Time) Resource creation Initiated 2024-03-17T20:21:19.223Z [INFO]:  2024-03-17T20:21:19.468Z [WARNING]: - Uploading files. 2024-03-17T20:21:19.760Z [WARNING]: - Uploading files. 2024-03-17T20:21:20.809Z [WARNING]: - Initializing your environment: saloma 2024-03-17T20:21:20.853Z [WARNING]: - Building resource api/platelet 2024-03-17T20:21:37.327Z [WARNING]: Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. 2024-03-17T20:21:37.335Z [WARNING]: Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. Be careful when using @auth directives on a field in a root type. @auth directives on field definitions use the source object to perform authorization logic and the source will be an empty object for fields on root types. Static group authorization should perform as expected. 2024-03-17T20:21:38.952Z [INFO]: āš ļø You are using the "pipelineFunctions" directory for overridden and custom resolvers. Please use the "resolvers" directory as "pipelineFunctions" will be deprecated. 2024-03-17T20:21:39.020Z [INFO]: āœ… GraphQL schema compiled successfully. Edit your schema at /codebuild/output/src1790987902/src/platelet/amplify/backend/api/platelet/schema.graphql or place .graphql files in a directory at /codebuild/output/src1790987902/src/platelet/amplify/backend/api/platelet/schema 2024-03-17T20:21:39.021Z [WARNING]: - Building resource auth/platelet61a0ac07 2024-03-17T20:21:42.123Z [WARNING]: - Building resource auth/userPoolGroups 2024-03-17T20:21:45.275Z [WARNING]: - Building resource storage/plateletStorage 2024-03-17T20:21:45.337Z [WARNING]: āœ” Initialized provider successfully. 2024-03-17T20:21:45.455Z [WARNING]: āœ– There was an error initializing your environment. 2024-03-17T20:21:45.455Z [INFO]: šŸ›‘ This environment is missing some parameter values. 2024-03-17T20:21:45.456Z [INFO]: [plateletDomainName,plateletWelcomeEmail,plateletDomainName,plateletWelcomeEmail,plateletDomainName,plateletWelcomeEmail,plateletSendFromEmailAddress,plateletSendToEmailAddress] do not have values. Resolution: Run 'amplify push' interactively to specify values. Alternatively, manually add values in SSM ParameterStore for the following parameter names: /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAddNewTenant_plateletDomainName /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAddNewTenant_plateletWelcomeEmail /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminAddNewUser_plateletDomainName /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminAddNewUser_plateletWelcomeEmail /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminResetUserPassword_plateletDomainName /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletAdminResetUserPassword_plateletWelcomeEmail /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletSendUserFeedback_plateletSendFromEmailAddress /amplify/dlkp8fs0a5r7s/saloma/AMPLIFY_function_plateletSendUserFeedback_plateletSendToEmailAddress Learn more at: https://docs.amplify.aws/cli/reference/ssm-parameter-store/#manually-creating-parameters 2024-03-17T20:21:45.493Z [INFO]: 2024-03-17T20:21:45.493Z [INFO]: Session Identifier: 787d8900-f996-4d45-8bb8-cbede870d338 2024-03-17T20:21:45.516Z [ERROR]: !!! Build failed 2024-03-17T20:21:45.516Z [ERROR]: !!! Error: Command failed with exit code 1 2024-03-17T20:21:45.517Z [INFO]: # Starting environment caching... 2024-03-17T20:21:45.517Z [INFO]: # Environment caching completed Terminating logging... ```

Additional information

No response

Before submitting, please confirm:

josefaidt commented 7 months ago

Hey @duckbytes :wave: thanks for raising this! While this approach is not officially supported for adding environment variables, you can get away with manually modifying amplify/team-provider-info.json and specifying key value pairs for your Function env vars. These will then be hydrated on init/checkout/pull and uploaded to SSM, which should resolve this issue.

Otherwise you can use amplify update function to CRUD env vars/secrets for your Functions https://docs.amplify.aws/react/build-a-backend/functions/environment-variables/

duckbytes commented 7 months ago

Thanks @josefaidt .

Originally I had not committed team-provider-info.json because the project is open source and it only seemed to contain data about my development environment. I'm also not working in a team.

Is using amplify/backend/function/<something>/parameters.json no longer a supported method?

josefaidt commented 7 months ago

Hey @duckbytes it is not an officially-supported method, but likely worked prior to the change of lifting those env-specific parameters to SSM Parameter Store (around CLI v9/v10). Those parameters/env vars are written to team-provider-info.json per env, and hydrated to backend-config.json as SSM Parameters on env checkout/pull. Those then flow to the Function on synth.

Despite the name, team-provider-info.json holds information relevant to each environment. If this file does not exist or we create a new env the CLI should prompt for env var values for each Function.

duckbytes commented 6 months ago

Hey @josefaidt. How would you recommend I maintain the team-provider-info.json file when working with multiple deployments? At the moment I have the Amplify console set to automatically create a deployment on new branches that match production/**.

What this meant is that I could create a new branch from master, push it up and the new deployment would be created for me without any interventions.

It sounds like now I will need to generate a new team-provider-info.json file every time I create a new deployment, which I guess means creating a new environment manually to put in those values, and then connecting the new environment to a branch afterwards. Or can I have one version of the file that is common to all branches and it'll pull those values? It seems like there is a lot of environment specific data held in that file, but those function parameters don't change between each deployment.

I'm just wondering how much of this I can automate, since it was working so well before.

ykethan commented 6 months ago

@duckbytes if Amplify CLI is unable to find values from team-provider-info.json file, it will fetch the values from SSM parameter store. Refer to the documentation providing this information. on a new branch creation you should be able to run a script on hosting preBuild phase that creates the SSM parameters for the branch as Amplify CLI tries to retrieve parameters with format /amplify/<appId>/<environment-name>/AMPLIFY_<category-name>_<resource-name>_<parameter-name>

duckbytes commented 6 months ago

Thanks @ykethan

In the end I've hard coded the values into the lambda function, since they aren't secrets and don't often change between deployments this seemed like the simplest solution.

github-actions[bot] commented 6 months ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.