aws-amplify / amplify-category-api

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development. This plugin provides functionality for the API category, allowing for the creation and management of GraphQL and REST based backends for your amplify project.
https://docs.amplify.aws/
Apache License 2.0
88 stars 76 forks source link

Query failing with "Condition parameter type does not match schema type" after Transformer 2.0 migration #789

Open parvusville opened 2 years ago

parvusville commented 2 years ago

Before opening, please confirm:

How did you install the Amplify CLI?

No response

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

10

What operating system are you using?

Pop os

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No

Amplify Categories

api

Amplify Commands

Not applicable

Describe the bug

Hello, I started receiving this error when using AMAZON_COGNITO_USER_POOLS auth on my query after migrating to GraphQL Transformer V2. If I remove categories from the query, it works. With API_KEY the query works as it should.

query MyQuery {
  listCompanies {
    items {
      id
      categories {
        items {
          id
        }
      }
    }
  }
}
{
  ...,
  "errorType": "DynamoDB:DynamoDbException",
  "message": "One or more parameter values were invalid: Condition parameter type does not match schema type (Service: DynamoDb, Status Code: 400, Request ID: 1E6IG5CR8J4B94VHOI3G3TMCONVV4KQNSO5AEMVJF66Q9ASUAAJG)"
}

Expected behavior

I should be able to include categories to my listCompanies query.

Reproduction steps

  1. amplify api migrate
  2. amplify push

GraphQL schema(s)

```graphql # My schema.graphql after migration: type Company @model @auth( rules: [ { allow: private, provider: iam } { allow: groups, groupsField: "group" } { allow: public, operations: [read] } ] ) { id: ID! categories: [Category] @hasMany(indexName: "byGroup", fields: ["group"]) ... } type Category @model @auth( rules: [ { allow: private, provider: iam } { allow: groups, groupsField: "group" } { allow: public, operations: [read] } ] ) { id: ID! group: String! @index(name: "byGroup") ... } # My schema.graphql before migration: type Company @model @auth( rules: [ { allow: private, provider: iam } { allow: groups, groupsField: "group" } { allow: public, operations: [read] } ] ) @key( name: "companyByGroup" fields: ["group"] queryField: "companyByGroup" ) { id: ID! categories: [Category] @connection(keyName: "byGroup", fields: ["group"]) } type Category @model @auth( rules: [ { allow: private, provider: iam } { allow: groups, groupsField: "group" } { allow: public, operations: [read] } ] ) @key(name: "byGroup", fields: ["group"]) { id: ID! group: String! ... } ```

Log output

``` # Put your logs below this line ```

Additional information

No response

josefaidt commented 2 years ago

Hey @parvusville :wave: thanks for raising this! It appears the attached schema is using the v1 transformer. Is this pre or post migration (where perhaps the auto-migration was not applied?)

parvusville commented 2 years ago

Hey @josefaidt . In the attached schema I have included both pre and post migration schemas.

josefaidt commented 2 years ago

@parvusville oh! Sorry I noticed the at-key 😷 taking a look

josefaidt commented 2 years ago

Hey @parvusville I was not able to reproduce this with a fresh v2 API, will try v1 migration next.

(ignore my admins group to create some companies)

type Company
  @model
  @auth(
    rules: [
      { allow: groups, groups: ["admins"] }
      { allow: groups, groupsField: "group" }
      { allow: public, operations: [read] }
    ]
  ) {
  id: ID!
  categories: [Category] @hasMany(indexName: "byGroup", fields: ["group"])
  group: String!
}

type Category
  @model
  @auth(
    rules: [
      { allow: groups, groups: ["admins"] }
      { allow: groups, groupsField: "group" }
      { allow: public, operations: [read] }
    ]
  ) {
  id: ID!
  group: String! @index(name: "byGroup")
}
image

Would you mind posting the resolvers that were generated for this list query?

parvusville commented 2 years ago

@josefaidt Here are resolvers for listCompanies and listCategories, hope these are what you were looking for. listCompanies:

``` ## Query.listCompanies.auth.1.req.vtl ## [Start] Authorization Steps. ** $util.qr($ctx.stash.put("hasAuth", true)) #set( $isAuthorized = false ) #set( $primaryFieldMap = {} ) #if( $util.authType() == "API Key Authorization" ) #set( $isAuthorized = true ) #end #if( $util.authType() == "IAM Authorization" ) #set( $adminRoles = ["my", "functions"] ) #foreach( $adminRole in $adminRoles ) #if( $ctx.identity.userArn.contains($adminRole) && $ctx.identity.userArn != $ctx.stash.authRole && $ctx.identity.userArn != $ctx.stash.unauthRole ) #return($util.toJson({})) #end #end #if( !$isAuthorized ) #if( ($ctx.identity.userArn == $ctx.stash.authRole) || ($ctx.identity.cognitoIdentityPoolId == "eu-west-1:redacted" && $ctx.identity.cognitoIdentityAuthType == "authenticated") ) #set( $isAuthorized = true ) #end #end #end #if( $util.authType() == "User Pool Authorization" ) #if( !$isAuthorized ) #set( $authFilter = [] ) #set( $role0 = $util.defaultIfNull($ctx.identity.claims.get("cognito:groups"), []) ) #if( $util.isString($role0) ) #if( $util.isList($util.parseJson($role0)) ) #set( $role0 = $util.parseJson($role0) ) #else #set( $role0 = [$role0] ) #end #end #if( !$role0.isEmpty() ) $util.qr($authFilter.add({ "group": { "in": $role0 } })) #end #if( !$authFilter.isEmpty() ) $util.qr($ctx.stash.put("authFilter", { "or": $authFilter })) #end #end #end #if( !$isAuthorized && $util.isNull($ctx.stash.authFilter) ) $util.unauthorized() #end $util.toJson({"version":"2018-05-29","payload":{}}) ## [End] Authorization Steps. ** ##Query.listCompanies.postAuth.1.req.vtl ## [Start] Sandbox Mode Disabled. ** #if( !$ctx.stash.get("hasAuth") ) $util.unauthorized() #end $util.toJson({}) ## [End] Sandbox Mode Disabled. ** ##Query.listCompanies.req.vtl ## [Start] List Request. ** #set( $args = $util.defaultIfNull($ctx.stash.transformedArgs, $ctx.args) ) #set( $limit = $util.defaultIfNull($args.limit, 100) ) #set( $ListRequest = { "version": "2018-05-29", "limit": $limit } ) #if( $args.nextToken ) #set( $ListRequest.nextToken = $args.nextToken ) #end #if( !$util.isNullOrEmpty($ctx.stash.authFilter) ) #set( $filter = $ctx.stash.authFilter ) #if( !$util.isNullOrEmpty($args.filter) ) #set( $filter = { "and": [$filter, $args.filter] } ) #end #else #if( !$util.isNullOrEmpty($args.filter) ) #set( $filter = $args.filter ) #end #end #if( !$util.isNullOrEmpty($filter) ) #set( $filterExpression = $util.parseJson($util.transform.toDynamoDBFilterExpression($filter)) ) #if( $util.isNullOrEmpty($filterExpression) ) $util.error("Unable to process the filter expression", "Unrecognized Filter") #end #if( !$util.isNullOrBlank($filterExpression.expression) ) #if( $filterExpression.expressionValues.size() == 0 ) $util.qr($filterExpression.remove("expressionValues")) #end #set( $ListRequest.filter = $filterExpression ) #end #end #if( !$util.isNull($ctx.stash.modelQueryExpression) && !$util.isNullOrEmpty($ctx.stash.modelQueryExpression.expression) ) $util.qr($ListRequest.put("operation", "Query")) $util.qr($ListRequest.put("query", $ctx.stash.modelQueryExpression)) #if( !$util.isNull($args.sortDirection) && $args.sortDirection == "DESC" ) #set( $ListRequest.scanIndexForward = false ) #else #set( $ListRequest.scanIndexForward = true ) #end #else $util.qr($ListRequest.put("operation", "Scan")) #end #if( !$util.isNull($ctx.stash.metadata.index) ) #set( $ListRequest.IndexName = $ctx.stash.metadata.index ) #end $util.toJson($ListRequest) ## [End] List Request. ** ##Query.listCompanies.res.vtl ## [Start] ResponseTemplate. ** #if( $ctx.error ) $util.error($ctx.error.message, $ctx.error.type) #else $util.toJson($ctx.result) #end ## [End] ResponseTemplate. ** ```

listCategories:

``` ## Query.listCategories.auth.1.req.vtl ## [Start] Authorization Steps. ** $util.qr($ctx.stash.put("hasAuth", true)) #set( $isAuthorized = false ) #set( $primaryFieldMap = {} ) #if( $util.authType() == "API Key Authorization" ) #set( $isAuthorized = true ) #end #if( $util.authType() == "IAM Authorization" ) #set( $adminRoles = ["my", "functions"] ) #foreach( $adminRole in $adminRoles ) #if( $ctx.identity.userArn.contains($adminRole) && $ctx.identity.userArn != $ctx.stash.authRole && $ctx.identity.userArn != $ctx.stash.unauthRole ) #return($util.toJson({})) #end #end #if( !$isAuthorized ) #if( ($ctx.identity.userArn == $ctx.stash.authRole) || ($ctx.identity.cognitoIdentityPoolId == "eu-west-1:redacted" && $ctx.identity.cognitoIdentityAuthType == "authenticated") ) #set( $isAuthorized = true ) #end #end #end #if( $util.authType() == "User Pool Authorization" ) #if( !$isAuthorized ) #set( $authFilter = [] ) #set( $role0 = $util.defaultIfNull($ctx.identity.claims.get("cognito:groups"), []) ) #if( $util.isString($role0) ) #if( $util.isList($util.parseJson($role0)) ) #set( $role0 = $util.parseJson($role0) ) #else #set( $role0 = [$role0] ) #end #end #if( !$role0.isEmpty() ) $util.qr($authFilter.add({ "group": { "in": $role0 } })) #end #if( !$authFilter.isEmpty() ) $util.qr($ctx.stash.put("authFilter", { "or": $authFilter })) #end #end #end #if( !$isAuthorized && $util.isNull($ctx.stash.authFilter) ) $util.unauthorized() #end $util.toJson({"version":"2018-05-29","payload":{}}) ## [End] Authorization Steps. ** ## Query.listCategories.postAuth.1.req.vtl ## [Start] Sandbox Mode Disabled. ** #if( !$ctx.stash.get("hasAuth") ) $util.unauthorized() #end $util.toJson({}) ## [End] Sandbox Mode Disabled. ** ## Query.listCategories.req.vtl ## [Start] List Request. ** #set( $args = $util.defaultIfNull($ctx.stash.transformedArgs, $ctx.args) ) #set( $limit = $util.defaultIfNull($args.limit, 100) ) #set( $ListRequest = { "version": "2018-05-29", "limit": $limit } ) #if( $args.nextToken ) #set( $ListRequest.nextToken = $args.nextToken ) #end #if( !$util.isNullOrEmpty($ctx.stash.authFilter) ) #set( $filter = $ctx.stash.authFilter ) #if( !$util.isNullOrEmpty($args.filter) ) #set( $filter = { "and": [$filter, $args.filter] } ) #end #else #if( !$util.isNullOrEmpty($args.filter) ) #set( $filter = $args.filter ) #end #end #if( !$util.isNullOrEmpty($filter) ) #set( $filterExpression = $util.parseJson($util.transform.toDynamoDBFilterExpression($filter)) ) #if( $util.isNullOrEmpty($filterExpression) ) $util.error("Unable to process the filter expression", "Unrecognized Filter") #end #if( !$util.isNullOrBlank($filterExpression.expression) ) #if( $filterExpression.expressionValues.size() == 0 ) $util.qr($filterExpression.remove("expressionValues")) #end #set( $ListRequest.filter = $filterExpression ) #end #end #if( !$util.isNull($ctx.stash.modelQueryExpression) && !$util.isNullOrEmpty($ctx.stash.modelQueryExpression.expression) ) $util.qr($ListRequest.put("operation", "Query")) $util.qr($ListRequest.put("query", $ctx.stash.modelQueryExpression)) #if( !$util.isNull($args.sortDirection) && $args.sortDirection == "DESC" ) #set( $ListRequest.scanIndexForward = false ) #else #set( $ListRequest.scanIndexForward = true ) #end #else $util.qr($ListRequest.put("operation", "Scan")) #end #if( !$util.isNull($ctx.stash.metadata.index) ) #set( $ListRequest.IndexName = $ctx.stash.metadata.index ) #end $util.toJson($ListRequest) ## [End] List Request. ** ## Query.listCategories.res.vtl ## [Start] ResponseTemplate. ** #if( $ctx.error ) $util.error($ctx.error.message, $ctx.error.type) #else $util.toJson($ctx.result) #end ## [End] ResponseTemplate. ** ```

For companies I also have other similar @hasMany fields, and all they work with user pool auth. For example I am able to query vehicles within listCompanies:

type Company
  id: ID!
  group: String! @index(name: "companyByGroup", queryField: "companyByGroup")
  vehicles: [Vehicle!]! @hasMany(indexName: "byCompany", fields: ["id"])
  categories: [Category] @hasMany(indexName: "byGroup", fields: ["group"])

type Vehicle
  @model
  @auth(
    rules: [
      { allow: private, provider: iam }
      { allow: groups, groupsField: "group" }
      { allow: public, operations: [read] }
    ]
  ) {
  id: ID!
  group: String! @index(name: "byGroup", queryField: "vehicleByGroup")
  companyId: ID! @index(name: "byCompany", queryField: "vehicleByCompany")

Then I also faced this issue: https://github.com/aws-amplify/amplify-category-api/issues/61 To me it seems that the Transformer v2 still has some severe isssues. Hoping these will get sorted out, but for now rolling back to v1. Until then fingers crossed that there won't be any further problems such as https://github.com/aws-amplify/amplify-category-api/issues/715 .

josefaidt commented 2 years ago

Hey @parvusville thank you for posting those! Would you mind sending the project ID from amplify diagnose --send-report?

parvusville commented 2 years ago

Is there another way to get the project ID @josefaidt ? My current 6.7.6 CLI version doesn't have that command, and I don't want to upgrade anything before I manage to create a new testing environment. I already got the dev blocked for days on this migration hustle. Everything seemed to be to be working after just the CLI upgrade though, so might try that later again if necessary. :sweat_smile:

parvusville commented 1 year ago

Hello @josefaidt . I'm now triyng the migration again in a new environment, and still facing the same problem. Here is the project identifier: 7b272ae9f13a888e244358c3d691228a

I also noticed, that after migration I receive the following error, if I try to listCompanies with IAM authorization: "message": "Not Authorized to access listCompanies on type ModelCompanyConnection"

E: The latter seems to happen on other models as well. As an example: "message": "Not Authorized to access listReservations on type ModelReservationConnection

josefaidt commented 1 year ago

Hey @parvusville apologies for the delay here! I was not able to reproduce an issue calling GraphQL from a Lambda function, and the resolver appears similar in that you also have the function names in the adminRoles list. Can you confirm the current environment's list* resolvers contains the correct function name (with environment)?

#if( $util.authType() == "IAM Authorization" )
  #set( $adminRoles = ["78954dd137f-dev"] )
  #foreach( $adminRole in $adminRoles )
    #if( $ctx.identity.userArn.contains($adminRole) && $ctx.identity.userArn != $ctx.stash.authRole && $ctx.identity.userArn != $ctx.stash.unauthRole )
      #return($util.toJson({}))
    #end
  #end
  #if( !$isAuthorized )
    #if( $ctx.identity.userArn == $ctx.stash.authRole )
      #set( $isAuthorized = true )
    #end
  #end
#end

Additionally, can you confirm your function's CloudFormation template includes a AmplifyResourcesPolicy resource that includes a reference to your GraphQL API?

"AmplifyResourcesPolicy": {
  "DependsOn": [
    "LambdaExecutionRole"
  ],
  "Type": "AWS::IAM::Policy",
  "Properties": {
    "PolicyName": "amplify-lambda-execution-policy",
    "Roles": [
      {
        "Ref": "LambdaExecutionRole"
      }
    ],
    "PolicyDocument": {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "appsync:GraphQL"
          ],
          "Resource": [
            {
              "Fn::Join": [
                "",
                [
                  "arn:aws:appsync:",
                  {
                    "Ref": "AWS::Region"
                  },
                  ":",
                  {
                    "Ref": "AWS::AccountId"
                  },
                  ":apis/",
                  {
                    "Ref": "api789GraphQLAPIIdOutput"
                  },
                  "/types/Query/*"
                ]
              ]
            },
            {
              "Fn::Join": [
                "",
                [
                  "arn:aws:appsync:",
                  {
                    "Ref": "AWS::Region"
                  },
                  ":",
                  {
                    "Ref": "AWS::AccountId"
                  },
                  ":apis/",
                  {
                    "Ref": "api789GraphQLAPIIdOutput"
                  },
                  "/types/Mutation/*"
                ]
              ]
            }
          ]
        }
      ]
    }
  }
}
parvusville commented 1 year ago

Hello @josefaidt , thanks for getting back to me.

The original problem is with Cognito. Any idea about that?

The problem I previously experienced with IAM Authorization was at AppSync console, and had to do with missing the custom-roles.json from my API. After adding that the listCompanies Query at my original post works with both API Key and IAM Authorization.

Besides these there is also this other active issue related to Cognito auth with dynamic groups: https://github.com/aws-amplify/amplify-category-api/issues/61

josefaidt commented 1 year ago

Hey @parvusville apologies for the delay here, and apologies for the confusion. Glad to hear the IAM issue is sorted out. Are you still experiencing the issue where although IAM and API key are able to fetch the nested categories from listCompanies, authorizing with Cognito User Pools is failing on this query?

While the error is not related to an authorization issue, I'm almost curious if this is related to how the group is referenced from the model's field:

#if( $util.authType() == "User Pool Authorization" )
  #if( !$isAuthorized )
    #set( $authFilter = [] )
    #set( $role0 = $util.defaultIfNull($ctx.identity.claims.get("cognito:groups"), []) )
    #if( $util.isString($role0) )
      #if( $util.isList($util.parseJson($role0)) )
        #set( $role0 = $util.parseJson($role0) )
      #else
        #set( $role0 = [$role0] )
      #end
    #end
    #if( !$role0.isEmpty() )
      $util.qr($authFilter.add({ "group": { "in": $role0 } }))
    #end
    #if( !$authFilter.isEmpty() )
      $util.qr($ctx.stash.put("authFilter", { "or": $authFilter }))
    #end
  #end
#end

Do you see this error regardless of whether the user is in the groups?

parvusville commented 1 year ago

Hello @josefaidt , yes I am sill experiencing the issue when using Cognito authentication. I receive the same error, regardless of if the user is in "ville" group or is not in group at all in this example. Below see how the "group" in response is a little different, when using Cognito authentication. Maybe that has something to do with this?

getCompany query: API KEY: "group": "ville" Cognito: "group": "[\"ville\"]"

Here are some example Queries I run in this Transformer v2 branch:

query MyQuery {
  companyByGroup(group: "ville") {
    items {
      id
      categories {
        items {
          id
        }
      }
    }
  }
  listCompanies {
    items {
      id
      categories {
        items {
          id
        }
      }
    }
  }
  getCompany(id: "e861d76e-cd62-4415-8b80-1eacd63249de") {
    id
    categories {
      items {
        id
      }
    }
    group
  }
}

Using API KEY: I only include getCompany, the responses are identical (I only have 1 Company in this environment):

{
  "data": {
    "getCompany": {
      "id": "e861d76e-cd62-4415-8b80-1eacd63249de",
      "categories": {
        "items": [
          {
            "id": "ae09e91f-768b-4dfd-a297-9fba2a7ef03c"
          }
        ]
      },
      "group": "ville"
    }
  }
}

Using Cognito: Notice how "group" in this response is different: "[\"ville\"]". In our dev branch using Transformer v1 the group returned is just "ville", as expected.

{
  "data": {
    "getCompany": {
      "id": "e861d76e-cd62-4415-8b80-1eacd63249de",
      "categories": null,
      "group": "[\"ville\"]"
    }
  },
  "errors": [
    {
      "path": [
        "getCompany",
        "categories"
      ],
      "data": null,
      "errorType": "DynamoDB:DynamoDbException",
      "errorInfo": null,
      "locations": [
        {
          "line": 4,
          "column": 5,
          "sourceName": null
        }
      ],
      "message": "One or more parameter values were invalid: Condition parameter type does not match schema type (Service: DynamoDb, Status Code: 400, Request ID: HLTQ4NFG2AFVDACN2M13NS2O5JVV4KQNSO5AEMVJF66Q9ASUAAJG)"
    }
  ]
} 
josefaidt commented 1 year ago

Hey @parvusville that issue in particular seems related to https://github.com/aws-amplify/amplify-category-api/issues/65, which has a few strategies for working around this such as overriding the associated resolver. Let me take a further look at this one to get a solid example to share

parvusville commented 1 year ago

Thanks @josefaidt , a solid example would be great. Seeing that these issues are over a year old however does make me a little worried of completing the migration. Especially when the workarounds come with a good amount of technical debt.