When creating any part of the composite sort key for @index '<my index>', you must provide all fields for the key. Missing key: 'createdAt'.

loganpowell commented 2 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

14.18.1

Amplify CLI Version

7.6.14

What operating system are you using?

Windows 10

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Amplify Categories

api

Amplify Commands

Not applicable

Describe the bug

When I have a composite index made with two sort keys, specifically createdAt, though this should be automatically handled by a PR made last year by Amplify, when I try to run a mutation, I'm required to add the createdAt field to the query.

I've set up a simple example repo that seems to work without having to require the createdAt key, but doesn't work in my project, which I'm currently trying to update to the v2 of the transformer.... perhaps that's the problem, IDK :(

I'm sorry I can't seem to reproduce the but using a simpler example.

Expected behavior

When using a composite sort key with createdAt, running a graphql mutation doesn't require it (being autogenerated by the appsync resolvers).

Reproduction steps

I'm sorry, but I tried to reproduce this with a simple Todo example, but it works there...

GraphQL schema(s)

```graphql # # # e88~~8e 888-~88e 888 888 888-~88e-~88e d88~\ # d888 88b 888 888 888 888 888 888 888 C888 # 8888__888 888 888 888 888 888 888 888 Y88b # Y888 , 888 888 888 888 888 888 888 888D # "88___/ 888 888 "88_-888 888 888 888 \_88P # # enum NodeType { # --- 🔻 REQUIRED: DO NOT REPLACE/REMOVE 🔻 --- R_ACCOUNT # --- 🔺 REQUIRED: DO NOT REPLACE/REMOVE 🔺 --- # Human ##################################### H_AUTHOR H_TEAM # Geographic Hierarchy ####################### # summary level = 040 (State) # GEO_01 # Alabama # ... # Thing ###################################### A_ARTICLE A_PAGE # findable by router/deeplink A_APPLICATION A_GEM A_COURSE # Survey S_ACS S_DECENNIAL S_CBP # Vintage V_1990 V_2000 V_2010 V_2020 # Collections/Groupings/branches ############# C_COURSES C_SERIES # Ordered/linked list C_LIST # Ordered/sortable } enum NodeStatus { DRAFT PRIVATE REVIEWED PUBLISHED EDITED DELETED } enum EdgeType { # Human to Asset AUTHORED # linked list/ordered HAS_NEXT HAS_PREVIOUS # Group to Subgroup HAS_PART # Node to Node: Hierarchical HAS_CHILD } enum AssetType { # Documentation ############################# """ A [description](http://spec.graphql.org/June2018/#sec-Descriptions), here in parentheses, is `markdown` friendly! """ DEPRECATED @deprecated(reason: "the reason is _also_ `markdown` friendly!") # Multimedia Assets ############################## A_IMAGE A_OG_IMAGE # open graph (https://ogp.me/#metadata) A_OG_AUDIO A_OG_VIDEO A_VIDEO A_AUDIO # Text (Markdown enabled) ####################### # open graph ⚠ needs client-side validation (custom UI) for optimal/max char length # optimal | max T_OG_TITLE # 55 | 95 T_OG_DESCRIPTION # 55 | 200 T_OG_TYPE T_LEDE T_BODY # Meta: each tag must be a separate resource-type in order for it to be able to be searched by M_DATA M_MAP M_VIZ M_API # Files ###################################### F_IMAGE F_AUDIO F_VIDEO F_PDF F_KML F_SHP F_CSV } # # d8 # /~~~8e d88~\ d88~\ e88~~8e _d88__ d88~\ # 88b C888 C888 d888 88b 888 C888 # e88~-888 Y88b Y88b 8888__888 888 Y88b # C888 888 888D 888D Y888 , 888 888D # "88_-888 \_88P \_88P "88___/ "88_/ \_88P # # interface Resource { id : ID! nodeID : ID! createdAt : AWSDateTime updatedAt : AWSDateTime type : AssetType! name : String! index : Int owner : String content : String editors : [String] } # Assets are sub-atomic, i.e., they are not - by themselves # - useful, but rather are combined into a Node, which is. # Nodes are the atomic unit of the system. If an Asset # (e.g., an image of a person) is needed for a different # use case than the Node that holds it (e.g., Node:Bio = # [ image, name, contact ]), then the Asset should be # copied to a new Node (e.g., Node:Author = [ image, name, # specialization_tags ]) type Asset implements Resource @model @auth(rules: [ { allow: owner, ownerField: "owner", identityClaim: "email" }, { allow: owner, operations: [ read, update ], ownerField: "editors", identityClaim: "email" }, { allow: groups, groups: ["Admins", "Editors"] }, { allow: groups, operations: [ read ], groups: ["Viewers"] }, { allow: public, operations: [ read ] provider: iam } ]) { id : ID! @primaryKey nodeID : ID! @index(name: "Assets_by_node", queryField: "assetsByNode") createdAt : AWSDateTime updatedAt : AWSDateTime type : AssetType! @index(name: "Assets_by_type", queryField: "assetsByType", sortKeyFields: ["createdAt"]) name : String! index : Int owner : String @index(name: "Assets_by_owner_type", queryField: "assetsByOwnerType", sortKeyFields: ["type", "createdAt"]) content : String editors : [String] } type AssetPr implements Resource @model @auth(rules: [ { allow: owner, ownerField: "owner", identityClaim: "email" }, { allow: owner, operations: [ read, update ], ownerField: "editors", identityClaim: "email" }, { allow: groups, groups: ["Admins", "Editors"] }, # https://docs.amplify.aws/cli/graphql-transformer/auth#owner-authorization ]) { id : ID! @primaryKey nodeID : ID! @index(name: "AssetsPr_by_node", queryField: "assetsPrByNode") createdAt : AWSDateTime updatedAt : AWSDateTime type : AssetType! # handles searching by type | type & createdAt @index(name: "AssetsPr_by_type", queryField: "assetsPrByType", sortKeyFields: ["createdAt"]) name : String! index : Int owner : String # handles searching by owner | owner & type | owner & type & createdAt @index(name: "AssetsPr_by_owner_type", queryField: "assetsPrByOwnerType", sortKeyFields: ["type", "createdAt"]) content : String editors : [String] } # # 888 # 888-~88e e88~-_ e88~\888 e88~~8e d88~\ # 888 888 d888 i d888 888 d888 88b C888 # 888 888 8888 | 8888 888 8888__888 Y88b # 888 888 Y888 ' Y888 888 Y888 , 888D # 888 888 "88_-~ "88_/888 "88___/ \_88P # # NOTES: # https://docs.amplify.aws/cli/graphql-transformer/key#designing-data-models-using-key # https://www.alexdebrie.com/posts/dynamodb-one-to-many/#composite-primary-key--the-query-api-action # https://docs.amplify.aws/cli/graphql-transformer/auth#field-level-authorization type Node @model @auth(rules: [ { allow: owner, ownerField: "owner", } # identityClaim: "email"}, { allow: groups, groups: ["Admins", "Editors"] }, { allow: groups, operations: [ read ], groups: ["Viewers"] }, { allow: public, operations: [ read ], provider: iam } ]) { id : ID! @primaryKey # handles searching by status | status & type | status & type & createdAt status : NodeStatus! @index(name: "Nodes_by_status_type_createdAt", queryField: "nodesByStatusType", sortKeyFields: ["type", "createdAt"]) type : NodeType! createdAt : AWSDateTime updatedAt : AWSDateTime owner : String # handles searching by owner | owner & status | owner & status & createdAt @index(name: "Nodes_by_owner_status_createdAt", queryField: "nodesByOwnerStatus", sortKeyFields: ["status", "createdAt"]) # handles searching by owner | owner & type | owner & type & createdAt @index(name: "Nodes_by_owner_type_createdAt", queryField: "nodesByOwnerType", sortKeyFields: ["type", "createdAt"]) assets : [Asset] @hasMany(indexName: "Assets_by_node", fields: ["id"]) assetsPr : [AssetPr] @hasMany(indexName: "AssetsPr_by_node", fields: ["id"]) edges : [Edge] @manyToMany(relationName: "EdgeNode") #edges : [EdgeNode] @hasMany(indexName: "EdgeNodes_by_node", fields: ["id"]) } # # 888 / # e88~~8e e88~\888 e88~88e e88~~8e d88~\ # d888 88b d888 888 888 888 d888 88b C888 # 8888__888 8888 888 "88_88" 8888__888 Y88b # Y888 , Y888 888 / Y888 , 888D # "88___/ "88_/888 Cb "88___/ \_88P # Y8""8D # type Edge @model @auth(rules: [ { allow: owner, ownerField: "owner" } # identityClaim: "email"}, { allow: groups, groups: ["Admins", "Editors"]}, { allow: groups, operations: [read], groups: ["Viewers"]}, { allow: public, operations: [ read ], provider: iam } ]) { id : ID! @primaryKey type : EdgeType! @index(name: "Edges_by_type", queryField: "edgesByType", sortKeyFields: ["createdAt"]) createdAt : AWSDateTime owner : String weight : Int nodes : [Node] @manyToMany(relationName: "EdgeNode") #nodes : [EdgeNode] @hasMany(indexName: "EdgeNodes_by_edge", fields: ["id"]) } ``` ```

Log output

``` # Put your logs below this line Not my logs, but here's an example resolver for one of the mutation functions (`MutationcreateAssetinit0Function `): ``` ## [Start] Initialization default values. ** $util.qr($ctx.stash.put("defaultValues", $util.defaultIfNull($ctx.stash.defaultValues, {}))) #set( $createdAt = $util.time.nowISO8601() ) $util.qr($ctx.stash.defaultValues.put("id", $util.autoId())) $util.qr($ctx.stash.defaultValues.put("createdAt", $createdAt)) $util.qr($ctx.stash.defaultValues.put("updatedAt", $createdAt)) $util.toJson({ "version": "2018-05-29", "payload": {} }) ## [End] Initialization default values. ** ``` ```

Additional information

No response

cowjen01 commented 2 years ago

@loganpowell Facing the same issue (even after migration to v2)

cjihrig commented 2 years ago

Here is a reduced schema that reproduces the issue:

input AMPLIFY { globalAuthRule: AuthRule = { allow: public } } # FOR TESTING ONLY!

type Todo @model {
  id: ID! @index(name: "gsi", queryField: "gsiQueryField", sortKeyFields: ["createdAt", "foobar"])
  createdAt: AWSDateTime
  foobar: ID
}

The createTodo mutation will fail with the error mentioned in the OP if the foobar field is provided, but createdAt is not. The reason is because the Mutation.createTodo.preAuth.1.req.vtl file only checks the inputs, and not any default/auto-computed values.

Marking as a bug. Thanks for reporting this.

ThomasRooney commented 2 years ago

Might be related: composite keys key1#key2 also cause models with field-level auth to fail interactions. Facing this after a migration from V1 to V2

E.g.

type StrippedExample @model
@auth(rules: [
    {allow: owner, operations: [create, update, read, delete]},
    {allow: owner, identityClaim: "custom:teamId", ownerField: "groupOwner", operations: [create, update, read, delete]},
    {allow: private, provider: iam, operations: [create, update, read, delete]}
])
{
    id: ID! @primaryKey
    startTime: AWSDateTime! @index(name: "ByStartTime")
    status: Status! @index(name: "ByStatus", queryField: "ByStatus", sortKeyFields: ["vmId", "startTime"]) @index(name: "ByStatusUpdatedAt", queryField: "ByStatusUpdatedAt", sortKeyFields: ["updatedAt"])
    vmId: String!
    runnerId: ID @auth(rules: [
        {allow: owner, operations: [read]},
        {allow: owner, identityClaim: "custom:teamId", ownerField: "groupOwner", operations: [read]},
        {allow: private, provider: iam, operations: [create, update, read, delete]}
    ])
    updatedAt: AWSDateTime!
    owner: String @index(name: "ByOwner")
    groupOwner: String @index(name: "ByGroupOwner", queryField: "ByGroupOwner")
}

Configures the composite field in Mutation.createStrippedExample.preAuth.2.req.vtl

...
#if( $hasSeenSomeKeyArg )
  $util.qr($ctx.args.input.put("vmId#startTime","${mergedValues.vmId}#${mergedValues.startTime}"))
#end
...

But doesn't include it in the allowedFields In Mutation.createStrippedExample.auth.1.req.vtl

#set( $inputFields = $util.parseJson($util.toJson($ctx.args.input.keySet())) )
...
  #set( $ownerAllowedFields0 = ["id","startTime","status","vmId","updatedAt","owner","groupOwner","_version","_deleted","_lastChangedAt"] )

Which then causes the graphql interaction to fail:

message: "Unauthorized on [vmId#startTime]"

From the later bit of Mutation.createStrippedExample.auth.1.req.vtl

...
  #set( $deniedFields = $util.list.copyAndRemoveAll($inputFields, $allowedFields) )
  #if( $deniedFields.size() > 0 )
    $util.error("Unauthorized on ${deniedFields}", "Unauthorized")
  #end
...

I suspect both issues result from composite fields being mishandled in some way; the pipeline resolver seems to add them to $ctx but forget them later.

adeldueast commented 5 months ago

Man this is still an issue occuring now !

aws-amplify / amplify-cli