Amplify failes with Attempting to edit the global secondary index in an unchanged manytomany table

bergmorten commented 2 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

7.6.26

What operating system are you using?

windows

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

None changes

Amplify Categories

api

Amplify Commands

publish

Describe the bug

We have not changed the graphQL scheme or any manually aws commands. The last build succeeded yesterday 22 march @ 17:30, but today build does not succeeds when pushing a simple web-code update that is not related to graphql scheme.

Trying to compile the graphQL localy complains with the same error: An illegal update of secondary keys. However this is a internal table generated by a @manyToMany relationship.

amplify push says that are no changes (and do nothing), but pushing code to Git that trigger backend build fails or locally trying amplify api gql-compile fails.

Expected behavior

Amplify CI and amplify api gql-compile should work when no changes are made to graphQL scheme

Reproduction steps

amplify api gql-compile

GraphQL schema(s)

```graphql type Fleet @model @auth( rules: [ { allow: private, operations: [read] } { allow: groups, groups: ["OrgAdmin"] } ] ) { id: ID! @primaryKey name: String! description: String users: [User] @manyToMany(relationName: "UserFleet") vehicles: [Vehicle] @hasMany(indexName: "byFleetVehicle", fields: ["id"]) projects: [Project] @manyToMany(relationName: "ProjectFleet") deployments: [Deployment] @hasMany(indexName: "byFleetDeployment", fields: ["id"]) logs: [Log] @hasMany(indexName: "byLogFleet", fields: ["id"]) } type Project @model @auth( rules: [ { allow: private, operations: [read] } { allow: groups, groups: ["OrgAdmin"] } ] ) { id: ID! @primaryKey name: String! description: String fleets: [Fleet] @manyToMany(relationName: "ProjectFleet") logs: [Log] @hasMany(indexName: "byLogProject", fields: ["id"]) } ```

Log output

`amplify pull --debug` ``` amplify pull --debug Pre-pull status: (node:13872) [DEP0128] DeprecationWarning: Invalid 'main' field in 'C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\cloudform\package.json' of 'packages/cloudform/index.js'. Please either fix that or report it to the module author (Use `node --trace-deprecation ...` to show where the warning was created) Current Environment: develop ┌──────────┬───────────────────────────┬───────────┬───────────────────┐ │ Category │ Resource name │ Operation │ Provider plugin │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Auth │ clientwebe913f698 │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Auth │ userPoolGroups │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Api │ customerAPI │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Api │ vehicleEmails │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Api │ vehicleCertificates │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Hosting │ amplifyhosting │ No Change │ │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ userCUDTrigger │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ userMigrationTrigger │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ vehicleEmailSender │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ preTokenTrigger │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ vehicleCertificateHandler │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ vehicleCommandHandler │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Storage │ customerData │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Custom │ SocketServerInterface │ No Change │ awscloudformation │ └──────────┴───────────────────────────┴───────────┴───────────────────┘ √ Successfully pulled backend environment develop from the cloud. Running generate components command in amplify-util-uibuilder Not pulling components because this project is not configured with the "react" framework GraphQL schema compiled successfully. Edit your schema at C:\Users\MortenBerg\Documents\Code\hefring-clients\amplify\backend\api\customerAPI\schema.graphql or place .graphql files in a directory at C:\Users\MortenBerg\Documents\Code\hefring-clients\amplify\backend\api\customerAPI\schema Successfully generated models. Generated models can be found in C:\Users\MortenBerg\Documents\Code\hefring-clients\src\clientApi Post-pull status: Current Environment: develop ┌──────────┬───────────────────────────┬───────────┬───────────────────┐ │ Category │ Resource name │ Operation │ Provider plugin │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Auth │ clientwebe913f698 │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Auth │ userPoolGroups │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Api │ customerAPI │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Api │ vehicleEmails │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Api │ vehicleCertificates │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Hosting │ amplifyhosting │ No Change │ │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ userCUDTrigger │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ userMigrationTrigger │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ vehicleEmailSender │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ preTokenTrigger │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ vehicleCertificateHandler │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Function │ vehicleCommandHandler │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Storage │ customerData │ No Change │ awscloudformation │ ├──────────┼───────────────────────────┼───────────┼───────────────────┤ │ Custom │ SocketServerInterface │ No Change │ awscloudformation │ └──────────┴───────────────────────────┴───────────┴───────────────────┘ ``` `amplify api gql-compile --debug` ``` amplify api gql-compile --debug (node:26104) [DEP0128] DeprecationWarning: Invalid 'main' field in 'C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\cloudform\package.json' of 'packages/cloudform/index.js'. Please either fix that or report it to the module author (Use `node --trace-deprecation ...` to show where the warning was created) 🛑 Attempting to edit the global secondary index byFleet on the ProjectFleetTable table in the ProjectFleet stack. InvalidGSIMigrationError: Attempting to edit the global secondary index byFleet on the ProjectFleetTable table in the ProjectFleet stack. at throwError (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\graphql-transformer-core\src\util\sanity-check.ts:148:11) at cantEditGSIKeySchemaRule (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\graphql-transformer-core\src\util\sanity-check.ts:191:9) at sanityCheckDiffs (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\graphql-transformer-core\src\util\sanity-check.ts:59:9) at sanityCheckProject (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\graphql-transformer-core\src\util\sanity-check.ts:41:21) at GraphQLSanityCheck.validate (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\amplify-provider-awscloudformation\src\graphql-transformer\sanity-check.ts:17:5) at buildAPIProject (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\amplify-provider-awscloudformation\src\graphql-transformer\transform-graphql-schema.ts:529:5) at transformGraphQLSchema (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\amplify-provider-awscloudformation\src\graphql-transformer\transform-graphql-schema.ts:394:29) at Object.executeAmplifyCommand (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\@aws-amplify\amplify-category-api\src\index.ts:224:5) at executePluginModuleCommand (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\src\execution-manager.ts:178:3) at executeCommand (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\src\execution-manager.ts:30:5) at Object.run (C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\src\index.ts:205:5) ```

Additional information

No response

bergmorten commented 2 years ago

Downgrading the amplify cli to prevous version seems to fix the problem.

npm remove -g @aws-amplify/cli npm i -g @aws-amplify/cli@7.6.25

amplify api gql-compile --debug       
(node:1524) [DEP0128] DeprecationWarning: Invalid 'main' field in 'C:\Users\MortenBerg\AppData\Roaming\npm\node_modules\@aws-amplify\cli\node_modules\cloudform\package.json' of 'packages/cloudform/index.js'. Please either fix that or report it to the module author
(Use `node --trace-deprecation ...` to show where the warning was created)
GraphQL schema compiled successfully.

marcvberg commented 2 years ago

I'm currently attempting to reproduce this issue - can you tell me which version of the CLI you were on before using 7.6.26? I created a @manyToMany relation on 7.6.24, pushed, upgrades to 7.6.26 and tried to run amplify api gql-compile but successfully completed without errors

bergmorten commented 2 years ago

I think my previous version also was 7.6.24, but do you have more than one manytomany connections?

Could it be that my cli.json has outdated settings, see my current setup below? This was the case with another bug.

{
  "features": {
    "graphqltransformer": {
      "addmissingownerfields": false,
      "improvepluralization": false,
      "validatetypenamereservedwords": true,
      "useexperimentalpipelinedtransformer": true,
      "enableiterativegsiupdates": false,
      "secondarykeyasgsi": false,
      "skipoverridemutationinputtypes": false,
      "transformerversion": 2,
      "suppressschemamigrationprompt": true,
      "securityEnhancementNotification": false,
      "showfieldauthnotification": false
    },
    "frontend-ios": {
      "enablexcodeintegration": false
    },
    "auth": {
      "enablecaseinsensitivity": false,
      "useinclusiveterminology": false,
      "breakcirculardependency": false,
      "forcealiasattributes": false,
      "useenabledmfas": false
    },
    "codegen": {
      "useappsyncmodelgenplugin": true,
      "usedocsgeneratorplugin": true,
      "usetypesgeneratorplugin": true,
      "cleangeneratedmodelsdirectory": true,
      "retaincasestyle": true,
      "addtimestampfields": true,
      "handlelistnullabilitytransparently": true,
      "emitauthprovider": false,
      "generateindexrules": false,
      "enabledartnullsafety": false
    },
    "appsync": {
      "generategraphqlpermissions": false
    },
    "latestregionsupport": {
      "pinpoint": 0,
      "translate": 0,
      "transcribe": 0,
      "rekognition": 0,
      "textract": 0,
      "comprehend": 0
    },
    "project": {
      "overrides": true
    }
  }
}

jdeeburke commented 2 years ago

@marcvberg We're seeing this too on 7.6.26 - Confirmed previous working version was 7.6.25.

Here's the first CloudFormation error that shows up in the build log: UPDATE_FAILED SermonTagTable AWS::DynamoDB::Table Wed Mar 23 2022 16:48:39 GMT+0000 (Coordinated Universal Time) Cannot update GSI's properties other than Provisioned Throughput and Contributor Insights Specification. You can create a new GSI with a different name.

And, here's the relevant excerpt of our schema.graphql (which hasn't changed recently):

type Sermon 
  @model 
  @auth(rules: [{ allow: owner }]) {
  id: ID!
  resourceId: String
  legacyId: String
  seriesId: ID @index(name: "bySeries")
  title: String!
  referenceVerses: String
  bigIdea: String
  blocks: AWSJSON
  isArchived: Boolean
  deletedAt: AWSDateTime
  sermonType: SermonType!
  description: String
  wordCount: Int
  owner: String @index(name: "byOwner")
  tags: [Tag] @manyToMany(relationName: "SermonTag")
}

type Tag 
  @model 
  @auth(rules: [{ allow: owner }]) {
  id: ID!
  title: String!
  sermons: [Sermon] @manyToMany(relationName: "SermonTag")
}

Confirmed that pinning our amplify-cli version to 7.6.25 brings us back to green builds.

marcvberg commented 2 years ago

My project uses "graphqltransformer"/"secondarykeyasgsi" set to true, as that's the default now. If you set that value to true does this issue persist?

bergmorten commented 2 years ago

Yes, we tried that, and still have the same issue.

7.6,25 works, so something must have changed in 7.6.26 regarding manytomany relationships.

On Thu, Mar 24, 2022 at 2:34 PM Marc VandenBerg @.***> wrote:

My project uses "graphqltransformer"/"secondarykeyasgsi" set to true, as that's the default now. If you set that value to true does this issue persist?

— Reply to this email directly, view it on GitHub https://github.com/aws-amplify/amplify-cli/issues/10051#issuecomment-1077634293, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUXMIRLEHBAKO7577FW6F5DVBRVNZANCNFSM5RNS3IRQ . You are receiving this because you authored the thread.Message ID: @.***>

josefaidt commented 2 years ago

Hey @bergmorten :wave: unfortunately we have not been successful in reproducing this issue and have a few follow-up questions:

do you also experience this issue using the latest version of Amplify CLI (8.0.1)
were there any manual changes made in the console? any GSI changes?
are you experiencing this issue upon executing amplify api gql-compile or on amplify push
using this doc as a reference, can you run a drift detection on both the parent CloudFormation stack and the affected CloudFormation stack?

bergmorten commented 2 years ago

I can confirm that the issue has been fixed in version 8.0.3 (not tested the previous ones).

josefaidt commented 2 years ago

Hey @bergmorten glad to hear you're back up and running! I have noted this issue as a point of reference when addressing push failures related to GraphQL API's, however for now I will close this issue. If this occurs again please do not hesitate to reply back to this thread and we can re-open to investigate further 🙂

vgribok commented 2 years ago

I am experiencing the "Attempting to edit the global secondary index [GSI name] on the [DDB table anem]Table table in the [stack name] stack." error with Amplify CLI v.10.4.1 - current latest - and a few other recent versions, when I do "amplify push". Running on Windows and having the same issue on two different systems.

aws-amplify / amplify-cli