ykethan
commented
2 years ago Hello @benjeater, Could you please provide us the team-provider-info.json.
Open benjeater opened 2 years ago
ykethan
commented
2 years ago Hello @benjeater, Could you please provide us the team-provider-info.json.
benjeater
commented
2 years ago @ykethan Here you go:
{
"dev": {
"awscloudformation": {
"AuthRoleName": "amplify-productname-dev-144015-authRole",
"UnauthRoleArn": "arn:aws:iam::465234467934:role/amplify-productname-dev-144015-unauthRole",
"AuthRoleArn": "arn:aws:iam::465234467934:role/amplify-productname-dev-144015-authRole",
"Region": "eu-west-1",
"DeploymentBucketName": "amplify-productname-dev-144015-deployment",
"UnauthRoleName": "amplify-productname-dev-144015-unauthRole",
"StackName": "amplify-productname-dev-144015",
"StackId": "arn:aws:cloudformation:eu-west-1:465234467934:stack/amplify-productname-dev-144015/48dc14b0-b1c1-11ec-a163-02844adf795f",
"AmplifyAppId": "doocypzu2klhi",
"AuthTriggerTemplateURL": "https://s3.amazonaws.com/amplify-productname-dev-144015-deployment/amplify-cfn-templates/auth/auth-trigger-cloudformation-template.json"
},
"categories": {
"auth": {
"productnameAuth": {}
},
"function": {
"productnameAuthCustomMessage": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameAuthCustomMessage-4e45586d677a55685432-build.zip"
},
"productnameAuthPostConfirmation": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameAuthPostConfirmation-706a4f5a4e476b474136-build.zip"
},
"productnameAuthPreTokenGeneration": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameAuthPreTokenGeneration-48503772736c44304d74-build.zip"
},
"productnameToolRunnerNMAP": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameToolRunnerNMAP-336e736e4a2f4a376139-build.zip"
},
"S3Triggere603b8f1": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/S3Triggere603b8f1-39736b5a6a3534775877-build.zip"
},
"productnameAuthPreSignup": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameAuthPreSignup-396e37724938546c7832-build.zip"
},
"productnameDBStreamEmployer": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameDBStreamEmployer-3157442f744c354d4343-build.zip"
},
"productnameToolRunnerHIBP": {
"secretsPathAmplifyAppId": "doocypzu2klhi",
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameToolRunnerHIBP-4a6e75354e57396b3839-build.zip"
},
"productnameToolRunnerOpenVAS": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameToolRunnerOpenVAS-63707866574b5a334e36-build.zip"
},
"productnameScanCreator": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameScanCreator-69526e4564554e59734d-build.zip"
},
"productnameCreateDailyStatistics": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameCreateDailyStatistics-792f6b4a527571665249-build.zip"
},
"productnameInitialSetup": {
"deploymentBucketName": "amplify-productname-dev-144015-deployment",
"s3Key": "amplify-builds/productnameInitialSetup-4139384561682f563169-build.zip"
}
},
"hosting": {
"amplifyhosting": {
"appId": "doocypzu2klhi",
"type": "cicd"
}
}
}
}
}hey @benjeater, Thank you for the information. looking at the file I observe that the demo branch is currently missing from the file. if we are trying to deploy a demo branch we will require values for this environment in the file..
Could you please try amplify env checkout <current-env-name> and see if the fields do get populated. Additionally, could you please try doing a amplify pull to pull in the latest changes from the project bucket.
benjeater
commented
2 years ago I first tried amplify checkout env demo
% amplify checkout env demo
Please pass in a valid environment name. Run amplify env list to get a list of valid environments
% amplify env list
| Environments |
| ------------ |
| *dev |There is no environment to switch to because the environment was created by connecting a branch from the GitHub repo to the project in the AWS Amplify console, not the CLI.
I then did a pull of the project to check the difference
% amplify pull
Pre-pull status:
Current Environment: dev
┌──────────┬───────────────────────────────┬───────────┬───────────────────┐
│ Category │ Resource name │ Operation │ Provider plugin │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productDBStreamEmployer │ Update │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productInitialSetup │ Update │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthCustomMessage │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthPostConfirmation │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthPreTokenGeneration │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ S3Triggere603b8f1 │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthPreSignup │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productToolRunnerNMAP │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productToolRunnerHIBP │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productToolRunnerOpenVAS │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productScanCreator │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productCreateDailyStatistics │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Auth │ productAuth │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Api │ productGraphQLAPI │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Hosting │ amplifyhosting │ No Change │ │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Storage │ productStorage │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Custom │ productErrorNotification │ No Change │ awscloudformation │
└──────────┴───────────────────────────────┴───────────┴───────────────────┘
Local changes detected.
Pulling changes from the cloud will override your local changes.
? Are you sure you would like to continue? Yes
⠹ Fetching updates to backend environment: dev from the cloud.Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
⠋ Building resource api/productGraphQLAPI✅ GraphQL schema compiled successfully.
Edit your schema at /Users/imac/git/product/amplify/backend/api/productGraphQLAPI/schema.graphql or place .graphql files in a directory at /Users/imac/git/product/amplify/backend/api/productGraphQLAPI/schema
Overrides functionality is not implemented for this category
✔ Successfully pulled backend environment dev from the cloud.
✅
Post-pull status:
Current Environment: dev
┌──────────┬───────────────────────────────┬───────────┬───────────────────┐
│ Category │ Resource name │ Operation │ Provider plugin │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Custom │ productErrorNotification │ Update │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthCustomMessage │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthPostConfirmation │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthPreTokenGeneration │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ S3Triggere603b8f1 │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productAuthPreSignup │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productToolRunnerNMAP │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productDBStreamEmployer │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productToolRunnerHIBP │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productToolRunnerOpenVAS │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productScanCreator │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productCreateDailyStatistics │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Function │ productInitialSetup │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Auth │ productAuth │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Api │ productGraphQLAPI │ No Change │ awscloudformation │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Hosting │ amplifyhosting │ No Change │ │
├──────────┼───────────────────────────────┼───────────┼───────────────────┤
│ Storage │ productStorage │ No Change │ awscloudformation │
└──────────┴───────────────────────────────┴───────────┴───────────────────┘The only differences after the pull are the datetimes of the last deployment (because it was built and deployed in the cloud it will have different timestamps from when I built it locally) and the deployment zip hashes.
I have another environment called e2etests which was previously deploying fine but has the same issue starting at the same time. I also created a brand new environment (newenvtest) to see if the problem happened with a new environment; it did and the second deployment did not work.
Your comment about the environment not being present in the team-provider-info.json file made me look for a way to pull this environment from the cloud. I found the Import an Environment process.
Using the Import an Environment process, populating the details from IAM, S3, and CloudFormation, I was able to add an environment to the team-provider-info.json file.
{
"dev": {
"awscloudformation": {
"AuthRoleName": "amplify-product-dev-144015-authRole",
"UnauthRoleArn": "arn:aws:iam::465234467934:role/amplify-product-dev-144015-unauthRole",
"AuthRoleArn": "arn:aws:iam::465234467934:role/amplify-product-dev-144015-authRole",
"Region": "eu-west-1",
"DeploymentBucketName": "amplify-product-dev-144015-deployment",
"UnauthRoleName": "amplify-product-dev-144015-unauthRole",
"StackName": "amplify-product-dev-144015",
"StackId": "arn:aws:cloudformation:eu-west-1:465234467934:stack/amplify-product-dev-144015/48dc14b0-b1c1-11ec-a163-02844adf795f",
"AmplifyAppId": "doocypzu2klhi",
"AuthTriggerTemplateURL": "https://s3.amazonaws.com/amplify-product-dev-144015-deployment/amplify-cfn-templates/auth/auth-trigger-cloudformation-template.json"
},
"categories": {
"auth": {
"productAuth": {}
},
"function": {
...
},
"hosting": {
"amplifyhosting": {
"appId": "doocypzu2klhi",
"type": "cicd"
}
}
}
},
"demo": {
"awscloudformation": {
"Region": "eu-west-1",
"DeploymentBucketName": "amplify-product-demo-70125-deployment",
"StackName": "amplify-product-demo-70125",
"StackId": "arn:aws:cloudformation:eu-west-1:465234467934:stack/amplify-product-demo-70125/9c670910-0279-11ed-9557-029315a1ff45",
"UnauthRoleName": "amplify-product-demo-70125-unauthRole",
"UnauthRoleArn": "arn:aws:iam::465234467934:role/amplify-product-demo-70125-unauthRole",
"AuthRoleName": "amplify-product-demo-70125-authRole",
"AuthRoleArn": "arn:aws:iam::465234467934:role/amplify-product-demo-70125-authRole"
}
}
}I then tried listing the environments in the CLI
% amplify env list
| Environments |
| ------------ |
| *dev |
| demo |Looks good so far I thought, so let's try switching to the demo environment:
% amplify checkout env demo
⠼ Initializing your environment: demo🛑 Could not initialize 'demo': Access DeniedIs there anything else I can try?
ykethan
commented
2 years ago Hello @benjeater, interesting. Could you please verify the profile being used for this environment by checking the local-aws-info.json present under the amplify/.config folder. If you are utilizing Vscode the folder may be hidden, please open the .vscode/settings.json and remove or comment out the line containing amplify/.config.
Could you please verify if the profile name does match with the profile in your credentials file locally and the credentials for the profile as well.
benjeater
commented
2 years ago @ykethan I can confirm that the settings in local-aws-info.json for the profileName match with a profile name available in my AWS credentials file. I can also confirm that the profile is able to perform amplify push on the dev environment (i.e. the locally created environment) without any issue.
ykethan
commented
2 years ago Hey @benjeater, to confirm by any chance do you have Branch auto detection enabled. For reference: https://docs.aws.amazon.com/amplify/latest/userguide/pattern-based-feature-branch-deployments.html
benjeater
commented
2 years ago Hey @ykethan; no, I am using the "connect branch" button on the app hosting environments page in the AWS console to create the new environments.
lazpavel
commented
2 years ago Hi @benjeater, from Amplify Console, can you go to your amplify project > backend environments > under Local Setup Instructions copy the amplify pull --appId <app-id> --envName demo and execute in an empty directory.
Are you getting any errors on pull? if not can you check that amplify env list prints the correct environment?
Can you run amplify diagnose --send-report and share the Project Identifier from the terminal?
benjeater
commented
2 years ago Hey @lazpavel, the project identifier is 0a2c62eccf8d3ac5d53ed152c8cb193b.
Running amplify pull --appId <app-id> --envName demo in an empty directory resulted in a successful pull of the environment data. I then ran an amplify pull on the dev environment in the same folder to ensure that an overwrite didn't occur. Everything seemed fine.
I then went back to my project directory, switched git to the demo branch and performed an amplify pull on the demo environment. This was all successful.
I performed an amplify push -y to update the demo environment in AWS; this was also successful.
The manual deployment resulted in local changes that I then committed to the demo branch of my GitHub repo. I performed a git commit and git push to the repo and the continuous deployment worked correctly.
It seems that this problem is isolated to environments that have never had a local amplify push (i.e. those created from repo branches with continuous deployment).
WORKAROUND:
amplify pull --appId <app-id> --envName <env-with-issue>
amplify push -yThis workaround unblocks me, but if I was using branch auto-detection, this wouldn't be a great solution. I'm not sure how much more help I can offer in narrowing this down, but let me know if there is anything else you want me to try.
ADDITIONAL TESTING:
I then switched to the new-env-test branch locally, made a small change to one of the files, and performed a git commit and git push. The continuous deployment triggered correctly and the deployment failed with the [secretsPathAmplifyAppId] must have values error. I therefore still have a branch that someone with more knowledge than me can take a look at if required.
ykethan
commented
2 years ago Hey @benjeater, Thank you for providing us the information. If I may ask what was the change performed on the files?
made a small change to one of the files
benjeater
commented
2 years ago @ykethan I bumped the package.json patch version
edwardfoyle
commented
2 years ago HI @benjeater for some reason it seems like you team-provider-info.json file is not being updated correctly when creating / checking out a new environment or not being persisted to your repo. Can you make sure that all your environments in the team-provider-info.json file have the secretsPathAmplifyAppId value defined and that these changes are committed to the repo? The file should look something like this:
{
"environment1": {
"awscloudformation": {
"AmplifyAppId": "abcd1234"
...
},
"categories": {
"function": {
"functionName": {
"secretsPathAmplifyAppId": "abcd1234"
}
}
}
},
"environment2": {
"awscloudformation": {
"AmplifyAppId": "9876qwer"
...
},
"categories": {
"function": {
"functionName": {
"secretsPathAmplifyAppId": "9876qwer"
}
}
}
}
}Make sure that the secretsPathAmplifyAppId is defined for all functions that use secrets and defined in all environments.
benjeater
commented
2 years ago Hey @edwardfoyle, I can confirm that the secretsPathAmplifyAppId exists in the dev environment of my team-provider-info.json file. 👍🏻
The environment created locally (dev) is fine, but this problem is for environments created via continuous deployment from my GitHub repo.
These environments never get written to the team-provider-info.json file because their creation and deployment is performed in the AWS cloud without any interaction with my local machine or a commit to the GitHub repo.
The initial deployment of the environment using continuous deployment works and has the right values (details in my initial post). The problem seems to be that between the initial deployment and the 2nd deployment the secret value is forgotten.
josefaidt
commented
2 years ago Reproduction steps:
amplify init > environment name maingit init; gh repo create; git commit -m 'init' --allow-empty; git push -u origin mainamplify add hosting > CI/CD with Amplify Hosting + Gitgit add .; git commit -m 'add hosting'; git pushgit checkout -b nextamplify env add > nextmysecret, with amplify add functionnext to mainCREATE_FAILED functionwithsecret AWS::CloudFormation::Stack Fri Jul 29 2022 15:03:54 GMT+0000 (Coordinated Universal Time) Parameters: [secretsPathAmplifyAppId] must have valuesMitigation steps:
main branch and main Amplify environmentteam-provider-info.json file is updated with the new function
"function": {
"withsecret": {}
}amplify update function we see this secret is associated with the function
Secrets configuration
- mysecretmain environment (note: previously we added this secret for the next environment, and the CLI will create an SSM parameter with the environment name in the key, which is why we need to step back through this flow for the main environment)
"function": {
"withsecret": {
"secretsPathAmplifyAppId": "du4677olxujg5"
}
}@benjeater would you say these reproduction steps accurately reflect the steps you took prior to receiving this error?
josefaidt
commented
2 years ago regression of https://github.com/aws-amplify/amplify-cli/issues/9667
benjeater
commented
2 years ago @josefaidt Reproduction steps 1-5 are the same, but step 6 is where my methodology differed. My steps would be:
amplify init > environment name maingit init; gh repo create; git commit -m 'init' --allow-empty; git push -u origin mainamplify add hosting > CI/CD with Amplify Hosting + Gitgit add .; git commit -m 'add hosting'; git pushgit checkout -b nextAll the same so far, but the following steps are my process:
mysecret, with amplify add functionnext branch to Amplify using the AWS consolenext branch locallynext environment, observe errorThis puts you in a state where the next environment is not stored in the team-provider-info.json file locally or in git because that environment was never created locally.
The mitigation process is to pull the next environment down from Amplify with the following commands:
amplify pull --appId <app-id> --envName <env-with-issue>
amplify push
git commit -m 'pull down of remote environment'
git pushThis gets the next environment details into the local team-provider-info.json file and means that subsequent commits to the branch in git will be deployed correctly. This mitigation process would probably be successful without the amplify push command because the commit to git would have the necessary information for a build.
If I was to guess where the problem is, it would be that an initial build environment of an environment created in Amplify console is different to subsequent pulls from GitHub. This difference means the initial build pulls data from the existing environment, but future builds use the details in the GitHub repo and can't find the details because the environment doesn't exist in the repo.
samputer
commented
2 years ago Running into the same problem, thanks to @benjeater 's work around of doing a pull first, I got this working on automatic branch deployments with the following added step in my amplify build settings:
The first command obtains the git branch name (which should match the backend name) and pulls the environment
- amplify pull --yes --appId $AWS_APP_ID --envName $AWS_BRANCH
- amplifyPush --simple
david-startinvest
commented
1 year ago Hello, I have the same issue. but the last solution you provided is not working for me. I did "amplify delete" and started amplify init , and created my functions from scratch and when I add hosting, I get same error as below. I have already added solution by @samputer to my build setting but that does not change. can someone help?
🛑 The following resources failed to deploy: Resource Name: functiondonationfinal3makepayment123 (AWS::CloudFormation::Stack) Event Type: update Reason: Parameters: [phrase, stripePkTest, stripeSkLive, stripePkLive, stripeSkTest] must have values URL: https://console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/arn%3Aaws%3Acloudformation%3Aus-west-1%3A733328664694%3Astack%2Famplify-donationfinal3-dev-171218%2F45fcfd60-bd4e-11ed-8ee4-02d69899c47f/events 2023-03-08T03:06:33.424Z [INFO]: 🛑 Resource is not in the state stackUpdateComplete Name: functiondonationfinal3makepayment123 (AWS::CloudFormation::Stack), Event Type: update, Reason: Parameters: [phrase, stripePkTest, stripeSkLive, stripePkLive, stripeSkTest] must have values Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/
david-startinvest
commented
1 year ago I forgot to mention that my environmental variables for function [phrase, stripePkTest, stripeSkLive, stripePkLive, stripeSkTest] they have value and are not empty. The project was working but after me tweaking functions with "aws udpate-function-config" I started getting error. even deleting the project and creating new amplify project is not helping .... I am not sure how Amplify is tracking me across various projects and same error triggers ...
BMscis
commented
3 months ago I had this issue after doing an amplify pull, for some reason, it sometimes removes the functions env variables and secrets. I was able to fix this by calling amplify update function and adding the secrets back.
Before opening, please confirm:
How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v16.13.0
Amplify CLI Version
9.1.0
What operating system are you using?
Mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No manual changes made
Amplify Categories
hosting
Amplify Commands
Not applicable
Describe the bug
I am using CI/CD connected to my GitHub repo. I have one of my branches (
dev) connected and the deployment is working correctly every time.I have connected a second branch (
demo) and the initial creation and deployment worked correctly.I then did a merge to the
demobranch from thedevbranch. The build phase failed with the error[secretsPathAmplifyAppId] must have valuesI have manually checked the following from the initial deployment of the
demobranch:demoenvironmentsecretsPathAmplifyAppIdis included in theteam-provider-info.jsonfile and has the correct valueI have checked for original issues regarding this issue and I have found #8513 and #9667, but these both seem to be on the first deploy, whereas this problem when using CI/CD appears to happen only on subsequent deployments.
Expected behavior
Projects using CI/CD hosting should deploy more than once.
Reproduction steps
[secretsPathAmplifyAppId] must have valuesin build logGraphQL schema(s)
Not applicable
Project Identifier
Project Identifier: 0a2c62eccf8d3ac5d53ed152c8cb193b
Log output
Additional information
No response