Closed cabcookie closed 1 month ago
I ran npx amplify sandbox
with the --debug
option and received more details:
[DEBUG] 2024-02-27T13:05:14.681Z:
❌ amplify-personalcrm-carskoch-sandbox-63e4bb09d0 failed: MissingRequiredParameter: Missing required key 'functionId' in params
at ParamValidator.fail (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:224807)
at ParamValidator.validateStructure (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:225157)
at ParamValidator.validateMember (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:225789)
at ParamValidator.validate2 [as validate] (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:224402)
at Request.VALIDATE_PARAMETERS (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:143072)
at Request.callListeners (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:91385)
at callNextListener (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:91177)
at /Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:141847
at finish (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:125042)
at /Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:125392 {
code: 'MissingRequiredParameter',
time: 2024-02-27T13:05:14.680Z
}
[DEBUG] 2024-02-27T13:05:14.681Z:
❌ Deployment failed: MissingRequiredParameter: Missing required key 'functionId' in params
at ParamValidator.fail (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:224807)
at ParamValidator.validateStructure (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:225157)
at ParamValidator.validateMember (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:225789)
at ParamValidator.validate2 [as validate] (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:224402)
at Request.VALIDATE_PARAMETERS (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:143072)
at Request.callListeners (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:91385)
at callNextListener (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:91177)
at /Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:141847
at finish (/Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:125042)
at /Users/carskoch/Development/personal-crm/node_modules/aws-cdk/lib/index.js:373:125392 {
code: 'MissingRequiredParameter',
time: 2024-02-27T13:05:14.680Z
}
Hey @cabcookie, could you try bumping the package version to beta which should contain the latest changes and re-run the sandbox.
"@aws-amplify/backend": "0.13.0-beta.3",
"@aws-amplify/backend-cli": "0.12.0-beta.3",
I created a separate sandbox to ensure I can reproduce the error if you come up with a recommendation. I just shifted back to the sandbox with the issue and tried to reproduce the error and yes I succeeded.
My simplified schema looks like this:
import { type ClientSchema, a, defineData } from "@aws-amplify/backend";
const schema = a.schema({
Activity: a
.model({
owner: a.string().authorization([a.allow.owner().to(["read", "delete"])]),
notionId: a.integer(),
notes: a.string(),
forProjects: a.manyToMany("Projects", {
relationName: "ProjectActivity",
}),
forMeeting: a.belongsTo("Meeting"),
finishedOn: a.datetime(),
})
.authorization([a.allow.owner()]),
});
export type Schema = ClientSchema<typeof schema>;
export const data = defineData({
schema,
authorizationModes: {
defaultAuthorizationMode: "userPool",
},
});
Now, when I add a property to my model (e.g., noteVersion: a.integer()
), I run into this issue.
I update my package.json with the following lines:
"@aws-amplify/backend": "0.13.0-beta.3",
"@aws-amplify/backend-cli": "0.12.0-beta.3",
and ran npm i
in my terminal. When I do the same schema change again it starts running the CloudFormation changes and succeeds.
I tried to understand what changed between the versions and can't spot what change could impact the sandbox to successfully making the changes now.
I removed the new property. The sandbox recognized the change on the schema and tried to implement the changes again and failed with the same error message:
The CloudFormation deployment has failed. Find more information in the CloudFormation AWS Console for this stack.
Caused By: ❌ Deployment failed: MissingRequiredParameter: Missing required key 'functionId' in params
As I seem to have a working sandbox I compared the 2 amplifyconfiguration.json
files and the main differences between the 2 were how the enums were treated.
When comparing the 2 configuration files, these are the differences (comparing the failing configuration to the working configuration):
Deleted:
// obviously as I added it for this test
Updated:
// obvious
// obvious
// obvious
Added:
I guess this is because of the "failing" configuration has been created with a previous version, and the new sandbox has been created using these versions:
@aws-amplify/backend: 0.12.1
@aws-amplify/backend-cli: 0.11.1
I have no idea if this impacts the behaviour and causing the problem though
Hey @cabcookie, thank you for providing this detailed information. I'm going to transfer this over to our API repository for better assistance 🙂.
Hey @cabcookie, We aim to explore this issue further. Could you possibly join a call with us to delve deeper into the issue?
Hey @cabcookie, Are you still facing this issue in the latest version?
@AnilMaktala See this issue with
"@aws-amplify/backend": "^1.0.2",
"@aws-amplify/backend-cli": "^1.0.3",
I saw this issue again today in my sandbox environment. Had to delete and recreate it. I am using the versions mentioned by @pkubat
... this?
Yes exactly @asanobm
@cabcookie
Yes, I've had that same experience multiple times. In the end, the only solutions were either to recreate the sandbox or reset the data structure to its original state and try again.
I apologize for not being of much help. It seems like there might be a bug. It would be a good idea to create a bug report.
npx ampx info
System:
OS: macOS 14.4.1
CPU: (10) arm64 Apple M1 Max
Memory: 22.71 GB / 64.00 GB
Shell: /bin/zsh
Binaries:
Node: 18.20.2 - ~/.nodebrew/current/bin/node
Yarn: undefined - undefined
npm: 10.8.0 - ~/workspace/fleeklounge/amplify-next-template/node_modules/.bin/npm
pnpm: undefined - undefined
NPM Packages:
@aws-amplify/backend: 1.0.2
@aws-amplify/backend-cli: 1.0.3
aws-amplify: 6.3.2
aws-cdk: 2.142.0
aws-cdk-lib: 2.142.0
typescript: 5.4.5
AWS environment variables:
AWS_STS_REGIONAL_ENDPOINTS = regional
AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables
@cabcookie
Hello. The same error occurred today as well. After trying various methods, I updated the amplify CLI and it seems to have resolved the issue.
@asanobm I seem to have gotten the same error today as well. Updating didn't seem to help.
I'm also bumping into this issue on: "@aws-amplify/backend": "^1.0.2", "@aws-amplify/backend-cli": "^1.0.3",
Also running into this issue
I got the same error. I reverted back to where there was no error, ran the sandbox command, and then I changed again. It worked for me. It was a weird behavior, but just give it a try.
The same error occurred as well. I commented out the model defined in resource.ts that I recently changed. After the error disappeared, I uncommented it, and it worked.
also seeing this issue with: "@aws-amplify/backend": "1.0.4", "@aws-amplify/backend-cli": "1.1.0",
Update 8/7: I found an underlying issue by running npx amplify sandbox
with --debug
and I have not encountered this again since resolving my user error.
Encountering this after schema modifications occasionally. I have to delete and re-seed my sandbox
Over the past 2 weeks. I ran into the issue multiple times. I couldn't find another solution than destroying my sandbox environment and creating a new one. I am running these versions right now:
System:
OS: macOS 14.5
CPU: (10) arm64 Apple M1 Pro
Memory: 148.27 MB / 32.00 GB
Shell: /bin/zsh
Binaries:
Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node
Yarn: undefined - undefined
npm: 10.2.3 - ~/.nvm/versions/node/v18.19.0/bin/npm
pnpm: undefined - undefined
NPM Packages:
@aws-amplify/backend: 1.0.3
@aws-amplify/backend-cli: 1.0.4
aws-amplify: 6.3.4
aws-cdk: 2.144.0
aws-cdk-lib: 2.144.0
typescript: 5.4.5
AWS environment variables:
AWS_STS_REGIONAL_ENDPOINTS = regional
AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables
Will update packages and try again.
I ran into this issue as well. In my case, I renamed an existing table field, and this error happened. Had to delete the sandbox and regenerate.
+1
As a result of a development process I've simply added an extra attribute to an already deployed model definition which resulted in above error. Even tho it's relatively easy to solve this in your sandbox as you remove then re-create your table but what would be a workaround in production type environment without losing data? Is this ever going to be fixed ?
Same here, added an attribute to my model, now the sandbox does not deploy anymore. Second time today. Hope this will get a fix soon.
Same here, added an attribute to my model, now the sandbox does not deploy anymore. Second time today. Hope this will get a fix soon.
Have you tried running your npx amplify sandbox
with --debug
? I've learned that this function error is very generic, and in my case I was able to get the details I needed to find the underlying issue by adding --debug
. Good luck!
I was able to reproduce the issue after restarting the sandbox and making changes to the model.
Run into this as well
I've removed "read" permission in auth rule and it started to fail with this functionId error
.authorization((allow) => [allow.owner().to("read")])
I'm not using amplify cli for deployments. I couldn't make it work for more standard "environments", rather than ephemeral sandboxes. It was conflicting with some environment-specific CDK resources.
So the deployment still works with a full deployment cycle, when CDK CLI has no hotswap
and direct
options. After a full update it can do hotswap again
If anybody wants to try it, this is the cdk command I'm using
cdk deploy --ci --all --output .amplify/artifacts/cdk.out --context amplify-backend-namespace=yourAmplifyAppId --context amplify-backend-name=master --require-approval never --context amplify-backend-type=branch --profile YourAwsSSOProfile-dev --hotswap-fallback --method=direct --watch --context secretLastUpdated=1718888347200 --debug
Calling the sandbox with npx ampx sandbox --debug
was a helpful hint. I even added a JSON.stringify(...)
[1] to the ./node_modules/aws-cdk/lib/index.js
as the error message Missing required key "functionId" in params
wasn't so helpful. The error comes up very randomly when I add a property in my table. I can't figure out what type of changes is causing the issue as the error messages are not providing enough details even in --debug
mode. With the JSON.stringify(...)
I added to the cdk code, I saw that one of the mutation functions of the model I changed was causing the error. This issue happens to me at least once a week. Now, at least I realized I can fix the issue without destroying the whole sandbox by just dropping this particular table and adding it to the schema again afterwards. Still I have to reproduce the data of that table and maybe the tables it has relations with but not the entire sandbox which is less time-consuming.
@treewhopper can you explain a bit how you identified the issue using the debug function? It might be super helpful to reproduce the error and fix it in my environment when it comes up or for the Amplify team to fix it entirely.
[1]
In the aws-cdk/lib/index.js
I searched for this code
this.fail("MissingRequiredParameter","Missing required key '"+paramName+"' in "+context)
and changed it to this
this.fail("MissingRequiredParameter","Missing required key '"+paramName+"' in "+context+": "+JSON.stringify({shape,params,context}))
to get more details on the error.
I haven't been able to repro this either. Can folks who have been able to repro confirm that they have completely wiped node_modules
, updated to latest versions of all amplify + aws + construct libs, and are still seeing the issue?
Just wanted to comment and say that I was having the same issue as everyone else.
failed: MissingRequiredParameter: Missing required key 'functionId'
I added a new field to the Composer model.
Composer: a
.model({
id: a.id().required(),
editionId: a.id().required(),
name: a.string().required(),
birthDate: a.string(),
deathDate: a.string(),
sex: a.string(),
gender: a.string(),
race: a.string(), //<--- This Field Was Added
country: a.string(),
//relationships && special aws data
sectionEntries: a.hasMany('SectionEntryComposer', ['composerId', 'composerEditionId']),
sectionEntryContents: a.hasMany('SectionEntryContentComposer', ['composerId', 'composerEditionId']),
books: a.hasMany('BookComposer', ['composerId', 'composerEditionId']),
createdAt: a.datetime(),
updatedAt: a.datetime(),
})
.identifier(['id', 'editionId'])
.secondaryIndexes((index) => [
index("editionId").queryField("listComposersByEditionId"),
])
.authorization(allow => [
allow.groups(['dev', 'admin']),
allow.groups(['staff']).to(['create', 'read', 'update']),
allow.authenticated().to(['read']),
]),
batchCreateComposers: a
.mutation()
.arguments({
editionId: a.id().required(),
items: a.json().required(),
})
.authorization(allow => [
allow.authenticated(),
])
.handler(a.handler.custom({
entry: './batch-create/Composer.js',
dataSource: a.ref('Composer'),
}))
.returns(a.json()),
Oddly enough, the table did update (I checked in DynamoDB), however, when I tried to run the listComposers query, it would tell me that the 'race' field I added (and could see in DynamoDB) didn't exist.
Anyway, these were my initial dependencies.
"devDependencies": {
"@aws-amplify/backend": "^1.0.4",
"@aws-amplify/backend-cli": "^1.2.2",
"aws-cdk": "^2.151.0",
"aws-cdk-lib": "^2.151.0",
"constructs": "^10.3.0"
}
And I updated them to:
"devDependencies": {
"@aws-amplify/backend": "^1.0.5",
"@aws-amplify/backend-cli": "^1.2.3",
"aws-cdk": "^2.153.0",
"aws-cdk-lib": "^2.153.0",
"constructs": "^10.3.0"
}
I reran the sandbox, and it seemed to work.
[DEBUG] 2024-08-20T16:51:33.285Z:
{
"compilerOptions": {
"target": "es2022",
"module": "es2022",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"strict": true,
"skipLibCheck": true,
"paths": {
"$amplify/*": [
"../.amplify/generated/*"
]
},
"allowSyntheticDefaultImports": true,
"resolvePackageJsonExports": true,
"resolvePackageJsonImports": true,
"useDefineForClassFields": true,
"noImplicitAny": true,
"noImplicitThis": true,
"strictNullChecks": true,
"strictFunctionTypes": true,
"strictBindCallApply": true,
"strictPropertyInitialization": true,
"alwaysStrict": true,
"useUnknownInCatchVariables": true
},
"files": [
"./backend.ts",
"./auth/resource.ts",
"./auth/post-confirmation/handler.ts",
"./auth/post-confirmation/resource.ts",
"./data/resource.ts",
"./data/add-user-to-group/handler.ts",
"./data/add-user-to-group/resource.ts",
"./data/remove-user-from-group/handler.ts",
"./data/remove-user-from-group/resource.ts"
]
}
I'm not sure if the issue still persists, but the next time I make a change, I'll come back and update this comment.
I still see this issue. Please see https://github.com/aws-amplify/amplify-category-api/issues/2770 for details on my reproduction.
It happened to me today.
I just added allow.authenticated().to(['create'])
to one of my tables.
This issue is still happening to me from time to time. I will walk you through what happened to me this morning and how I fixed it without recreating the whole sandbox environment.
I have a schema with these 2 models:
RelationshipType: a
.model({
owner: a
.string()
.authorization((allow) => [allow.owner().to(["read", "delete"])]),
name: a.string().required(),
people: a.hasMany("PersonRelationship", "typeId"),
})
.authorization((allow) => [allow.owner()]),
PersonRelationship: a
.model({
owner: a
.string()
.authorization((allow) => [allow.owner().to(["read", "delete"])]),
personId: a.id().required(),
person: a.belongsTo("Person", "personId"),
typeId: a.id().required(),
type: a.belongsTo("RelationshipType", "typeId"),
relatedPersonId: a.id(),
relatedPerson: a.belongsTo("Person", "relatedPersonId"),
date: a.date(),
})
.authorization((allow) => [allow.owner()]),
I wanted to add the property nameOfAnniversary
to the RelationshipType
model and when the sandbox environment tried to apply the change it ran into the issue:
The CloudFormation deployment has failed.
Caused By: Deployment failed: MissingRequiredParameter: Missing required key 'functionId' in params
Resolution: Find more information in the CloudFormation AWS Console for this stack.
This is how my model looks like after the change:
RelationshipType: a
.model({
owner: a
.string()
.authorization((allow) => [allow.owner().to(["read", "delete"])]),
name: a.string().required(),
people: a.hasMany("PersonRelationship", "typeId"),
nameOfAnniversary: a.string(), // this is the property I added
})
.authorization((allow) => [allow.owner()]),
PersonRelationship: a
.model({
owner: a
.string()
.authorization((allow) => [allow.owner().to(["read", "delete"])]),
personId: a.id().required(),
person: a.belongsTo("Person", "personId"),
typeId: a.id().required(),
type: a.belongsTo("RelationshipType", "typeId"),
relatedPersonId: a.id(),
relatedPerson: a.belongsTo("Person", "relatedPersonId"),
date: a.date(),
})
.authorization((allow) => [allow.owner()]),
Now, to ensure I do not need to recreate the whole sandbox I just removed the affected model and its relations:
// RelationshipType: a
// .model({
// owner: a
// .string()
// .authorization((allow) => [allow.owner().to(["read", "delete"])]),
// name: a.string().required(),
// people: a.hasMany("PersonRelationship", "typeId"),
// nameOfAnniversary: a.string(),
// })
// .authorization((allow) => [allow.owner()]),
PersonRelationship: a
.model({
owner: a
.string()
.authorization((allow) => [allow.owner().to(["read", "delete"])]),
personId: a.id().required(),
person: a.belongsTo("Person", "personId"),
typeId: a.id().required(),
// type: a.belongsTo("RelationshipType", "typeId"),
relatedPersonId: a.id(),
relatedPerson: a.belongsTo("Person", "relatedPersonId"),
date: a.date(),
})
.authorization((allow) => [allow.owner()]),
The sandbox environment applied the changes. After that, I re-introduced the model and its relations and the change went through successfully. Because of these changes I only lost data in the PersonRelationship
and the RelationshipType
models and not my whole sandbox environment.
I hope this helps some of you to reduce the pain. Still, it would be nice if this bug gets fixed at some point.
Hey @cabcookie, Thank you for providing the additional information. We are actively investigating this issue and will update the ticket as soon as we have a resolution.
I've been investigating this issue trying to find the root cause. It appears the issue only affects models with group or owner auth. If you have any other findings please to continue to update this issue.
I have not been able to identify the root cause of the issue yet, but I am now confident this issue will only arise when using CDK hotswap. That means this issue will not occur in production environment and is limited to sandbox.
The current work around is to recreate the sandbox or remove the table causing the issue and then restore the table. We recognize that this workaround isn't ideal and we are continuing to investigate the root cause.
The current work around is to recreate the sandbox or remove the table causing the issue and then restore the table. We recognize that this workaround isn't ideal and we are continuing to investigate the root cause.
A better workaround is to update stack without hotswap option, which fixes the problem and then hot swaps continue to work, I described above. However, amplify doesn't expose that functionality via native CDK to the users, which I think is a missing feature.
If users had option to make cdk updates via native cdk cli, it'd opened additional options for development. My use case is a permanent dev environments. I have some extra resources that take forever to create (dns, ses, acm), thus default amplify sandboxes is not ideal.
I would make native use of cdk cli official, there is not much to configure. The only real problem is credential update time cycle.
When i run into this issue it is enough to just comment out one of my custom resolvers, deploy, add it back in and deploy again instead of removing a table or even recreating the sandbox, which is much more cumbersome.
Hi @cabcookie and @thomasoehri , This issue has been resolved in https://github.com/aws/aws-cdk/releases/tag/v2.159.0 version. Please upgrade your lib to v2.159 version and let us know the results.
This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.
This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.
Environment information
Description
Hello, I run into an issue with my sandbox environment. It happened twice to me in the last 2 weeks and I had to delete and recreate my sandbox which has been quite cumbersome (including recreation of demo data). I login to my account using SSO and the profile org-data-amplify. My session is valid for a couple of hours and then I need to sign in again. Yesterday, I launched my sandbox environment with
npx amplify sandbox --profile org-data-amplify
and forgot to stop it. So, today I got back to VS Code (I didn't restart it) and stopped the sandbox and ran the command again. Now, I tried to make a change to my data model. I changed my Activity schema fromto
Amplify identified correctly a file update, synthesized the template and tried to deploy the update but then failed with the mentioned error message. Here is the output in my console: [Sandbox] Triggered due to a file update event: /Users/carskoch/Development/personal-crm/amplify/data/resource.ts
✨ Synthesis time: 3.86s
⚠️ The --hotswap and --hotswap-fallback flags deliberately introduce CloudFormation drift to speed up deployments ⚠️ They should only be used for development - never use them for your production Stacks!
amplify-personalcrm-carskoch-sandbox-63e4bb09d0: deploying... [1/1]
✨ hotswapping resources: ✨ AWS::AppSync::FunctionConfiguration 'MutationcreateActivityauth0Function' ✨ AWS::AppSync::FunctionConfiguration 'MutationupdateActivityauth0Function' ✨ AWS::AppSync::GraphQLSchema 'uevvie4lgje2pegsontu5uvbwaGraphQLSchema' ✨ Contents of S3 Bucket 'amplify-personalcrm-carsk-amplifydataamplifycodege-8slq063fvosf'
✨ Contents of S3 Bucket 'amplify-personalcrm-carsk-amplifydataamplifycodege-8slq063fvosf' hotswapped! ✨ AWS::AppSync::GraphQLSchema 'uevvie4lgje2pegsontu5uvbwaGraphQLSchema' hotswapped! The CloudFormation deployment has failed. Find more information in the CloudFormation AWS Console for this stack. Caused By: ❌ Deployment failed: MissingRequiredParameter: Missing required key 'functionId' in params
It mentions I should verify CloudFormation. However, CloudFormation is not showing any events. How can I fix this issue?