AfterChange Collection Hook shows Validation Error that prevents updating another collection with created doc.id

[miiketran]

Link to reproduction

No response

Describe the Bug

New bug introduced in Payload 2.0 where the afterChange collection hook does not have knowledge about the created doc. I see a Validation Error when I try to update another collection with the recently created document id. Error states it is an invalid selection.

To Reproduce

This flow worked in my previous version before upgrading from 1.13.2 to latest payload 2.3.1.

Background: I have a stores collection with a field name: "foods" with type: "relationship" and relationTo: "foods". I have a foods collection with a field name: "storeId" with a type: relationship" and relationTo: "stores" After I create a new food, I have an afterChange hook "addFoodToStore". I get the doc.id (since the food was created and I have the id) and push it to the newFoods array where I will payload.update collection "stores" with the newFoods.

However, I keep getting the validation error:

[00:08:22] ERROR (payload): The following field is invalid: foods
    err: {
      "type": "ValidationError",
      "message": "The following field is invalid: foods",
      "stack":
          ValidationError: The following field is invalid: foods
              at beforeChange (.../node_modules/payload/src/fields/hooks/beforeChange/index.ts:60:11)
              at processTicksAndRejections (node:internal/process/task_queues:95:5)
              at updateByID (.../node_modules/payload/src/collections/operations/updateByID.ts:230:18)
      "data": [
        {
          "field": "foods",
          "message": "This field has the following invalid selections: \"656e69f69002dcffc8e7847e\", "
        }
      ],
      "isOperational": true,
      "isPublic": false,
      "status": 400,
      "name": "ValidationError"
    }
[00:08:23] ERROR (payload): ValidationError: The following field is invalid: foods
    at beforeChange (.../node_modules/payload/src/fields/hooks/beforeChange/index.ts:60:11)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at updateByID (.../node_modules/payload/src/collections/operations/updateByID.ts:230:18)
       "field": "foods",
          "message": "This field has the following invalid selections: \"656e69f69002dcffc8e7847e\", "
        }
      ],
      "isOperational": true,
      "isPublic": false,
      "status": 400,
      "name": "ValidationError"
    }

If I move back to Payload version 1.13.2, the issue is no longer there. I have moved to Payload 2.0 but this is a major change that prevents me from adopting.

My analysis is that when req.payload.update is called to update the stores collection with newFoods, the food actually has not been created. I even tried running req.payload.update food collection with the doc.id and it said id did not exist.

Therefore, afterChange is not functioning after the document is created as it states in the docs even though there is an id for the doc object. After the error appears, I do not see a new food record so I suspect must be something wrong w/ the afterChange hook.

Payload Version

2.3.1

Adapters and Plugins

No response

[jacobsfletch]

Hey @miiketran this appears to be working for me, so I wonder if there's something else going on with your config. I did attempt to reproduce your issue on this branch but only ended up with a working proof of concept. Here's what my hook looks like:

{
  // Foods.ts
  // ...
  hooks: {
    afterChange: [
      async ({ doc, req }) => {
        if (doc.store) {
          try {
            const storeID = doc.store && typeof doc.store === 'object' ? doc.store.id : doc.store

            // Add food to store
            const store: Store = await req.payload.findByID({
              collection: 'stores',
              id: storeID,
            })

            const newFoods = [...(store?.foods || []), doc.id]

            await req.payload.update({
              collection: 'stores',
              id: storeID,
              data: {
                foods: newFoods,
              },
            })
          } catch (error) {
            req.payload.logger.error(error)
          }
        }
      },
    ],
  },
  // ...
}

Would you be able to provide some more insight into this? Or better yet, pull down this branch and continue try to and reproduce this in the _community directory.

[creekdrops]

I too am experiencing this on 2.4.0 with postgres. You can see this in action at my test repo.

I have an employees collection and a sorting global. I've created a hook that SHOULD populate the sorting collection with a new employee relationship when a new employee is created. Unfortunately, when executing the hook, the following error is thrown:

error: insert or update on table "sorting_rels" violates foreign key constraint "sorting_rels_employees_id_employees_id_fk"
    at ... {
  length: 312,
  severity: 'ERROR',
  code: '23503',
  detail: 'Key (employees_id)=(19) is not present in table "employees".',
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: 'public',
  table: 'sorting_rels',
  column: undefined,
  dataType: undefined,
  constraint: 'sorting_rels_employees_id_employees_id_fk',
  file: 'ri_triggers.c',
  line: '2608',
  routine: 'ri_ReportViolation'
}

From what I can tell, it seems like the hook is trying to update the global before the new employee record is saved to the database.

For some additional context, my hook is defined like so:

export const addToSortedHook: CollectionAfterChangeHook = async ({
  doc,
  operation,
  req,
}) => {
  // Only run this hook when creating a new employee
  if (operation === "create" && doc) {
    try {
      const { id } = doc;

      const employeeId = parseInt(id);

      const { employeeOrder } = await req.payload.findGlobal({
        slug: "sorting",
        depth: 0,
      });

      // BUG: It currently appears that the employees table has not been updated with
      // the new employee record at the time this hooks is run.

      await req.payload.updateGlobal({
        slug: "sorting",
        data: {
          employeeOrder: [...employeeOrder, employeeId],
        },
        overrideAccess: true,
      });
    } catch (error) {
      console.log(error);
    }
  }
};

My employee collection can be seen here:

const Employees: CollectionConfig = {
  slug: "employees",
  fields: [
    slug("name"),
    {
      name: "name",
      type: "text",
      required: true,
    },
    {
      name: "createdBy",
      type: "relationship",
      relationTo: "users",
      admin: {
        position: "sidebar",
        readOnly: true,
        condition: (data) => data.createdBy,
      },
    },
    {
      name: "title",
      type: "text",
      required: true,
    },
    {
      name: "bio",
      type: "richText",
    },
    {
      name: "image",
      type: "upload",
      relationTo: "media",
    },
  ],
  hooks: {
    beforeChange: [setCreatedByUserIdHook],
    afterChange: [addToSortedHook],
  },
  access: {
    read: () => true,
    create: isAdmin,
    update: isAdmin,
    delete: isAdmin,
  },
  admin: {
    useAsTitle: "name",
    defaultColumns: ["name", "title"],
  },
  versions: {
    drafts: true,
  },
};

And my global Sorting config is defined like so:

const Sorting: GlobalConfig = {
  slug: "sorting",
  fields: [
    {
      type: "tabs",
      tabs: [
        {
          label: "Employees",
          fields: [
            {
              type: "relationship",
              relationTo: "employees",
              name: "employeeOrder",
              hasMany: true,
            },
          ],
        },
      ],
    },
  ],
  access: {
    read: () => true,
    update: isAdmin,
  },
  admin: {
    group: "Site Settings",
  },
};

[miiketran]

Hi @jacobsfletch , thanks for looking into it. I'm having trouble running the _community directory locally so I'm not able to test.

To confirm, in your working proof of concept, you say you're successfully able to update the stores collection with the newFoods? Because your hook looks like it is doing the simple task that I'm intending. I don't know if there is more insight I can provide besides me receiving a validation error when trying to update stores with newFoods.

I believe @creekdrops is illustrating the same issue as mine. We can switch to looking into their issue and I think it may also resolve mine.

[kareem717]

I too am experiencing this on 2.4.0 with postgres. You can see this in action at my test repo.

I have an employees collection and a sorting global. I've created a hook that SHOULD populate the sorting collection with a new employee relationship when a new employee is created. Unfortunately, when executing the hook, the following error is thrown:

error: insert or update on table "sorting_rels" violates foreign key constraint "sorting_rels_employees_id_employees_id_fk"
    at ... {
  length: 312,
  severity: 'ERROR',
  code: '23503',
  detail: 'Key (employees_id)=(19) is not present in table "employees".',
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: 'public',
  table: 'sorting_rels',
  column: undefined,
  dataType: undefined,
  constraint: 'sorting_rels_employees_id_employees_id_fk',
  file: 'ri_triggers.c',
  line: '2608',
  routine: 'ri_ReportViolation'
}

From what I can tell, it seems like the hook is trying to update the global before the new employee record is saved to the database.

For some additional context, my hook is defined like so:

export const addToSortedHook: CollectionAfterChangeHook = async ({
  doc,
  operation,
  req,
}) => {
  // Only run this hook when creating a new employee
  if (operation === "create" && doc) {
    try {
      const { id } = doc;

      const employeeId = parseInt(id);

      const { employeeOrder } = await req.payload.findGlobal({
        slug: "sorting",
        depth: 0,
      });

      // BUG: It currently appears that the employees table has not been updated with
      // the new employee record at the time this hooks is run.

      await req.payload.updateGlobal({
        slug: "sorting",
        data: {
          employeeOrder: [...employeeOrder, employeeId],
        },
        overrideAccess: true,
      });
    } catch (error) {
      console.log(error);
    }
  }
};

My employee collection can be seen here:

const Employees: CollectionConfig = {
  slug: "employees",
  fields: [
    slug("name"),
    {
      name: "name",
      type: "text",
      required: true,
    },
    {
      name: "createdBy",
      type: "relationship",
      relationTo: "users",
      admin: {
        position: "sidebar",
        readOnly: true,
        condition: (data) => data.createdBy,
      },
    },
    {
      name: "title",
      type: "text",
      required: true,
    },
    {
      name: "bio",
      type: "richText",
    },
    {
      name: "image",
      type: "upload",
      relationTo: "media",
    },
  ],
  hooks: {
    beforeChange: [setCreatedByUserIdHook],
    afterChange: [addToSortedHook],
  },
  access: {
    read: () => true,
    create: isAdmin,
    update: isAdmin,
    delete: isAdmin,
  },
  admin: {
    useAsTitle: "name",
    defaultColumns: ["name", "title"],
  },
  versions: {
    drafts: true,
  },
};

And my global Sorting config is defined like so:

const Sorting: GlobalConfig = {
  slug: "sorting",
  fields: [
    {
      type: "tabs",
      tabs: [
        {
          label: "Employees",
          fields: [
            {
              type: "relationship",
              relationTo: "employees",
              name: "employeeOrder",
              hasMany: true,
            },
          ],
        },
      ],
    },
  ],
  access: {
    read: () => true,
    update: isAdmin,
  },
  admin: {
    group: "Site Settings",
  },
};

I am experiencing the same issue with a very similar config. I think this might have something to do with the postgres db adapter.

[timoconnellaus]

@jacobsfletch I was able to reproduce the error using your example

I created a repo using your config and the create payload cli. I also updated some of the package versions

The repo: https://github.com/timoconnellaus/payload-reproduce-error

[05:31:10] ERROR (payload): The following field is invalid: foods err: { "type": "ValidationError", "message": "The following field is invalid: foods", "stack": ValidationError: The following field is invalid: foods at beforeChange (/Users/tim/repos/payload-reproduce-error/node_modules/.pnpm/payload@2.4.0_typescript@4.8.4_webpack@5.89.0/node_modules/payload/src/fields/hooks/beforeChange/index.ts:60:11) at processTicksAndRejections (node:internal/process/task_queues:95:5) at updateByID (/Users/tim/repos/payload-reproduce-error/node_modules/.pnpm/payload@2.4.0_typescript@4.8.4_webpack@5.89.0/node_modules/payload/src/collections/operations/updateByID.ts:230:18) "data": [ { "field": "foods", "message": "This relationship field has the following invalid relationships: [object Object] 0" } ], "isOperational": true, "isPublic": false, "status": 400, "name": "ValidationError" }

[JarrodMFlesch]

@timoconnellaus

I created a repo using your config and the create payload cli. I also updated some of the package versions

The repo: https://github.com/timoconnellaus/payload-reproduce-error

[05:31:10] ERROR (payload): The following field is invalid: foods err: { "type": "ValidationError", "message": "The following field is invalid: foods", "stack": ValidationError: The following field is invalid: foods at beforeChange (/Users/tim/repos/payload-reproduce-error/node_modules/.pnpm/payload@2.4.0_typescript@4.8.4_webpack@5.89.0/node_modules/payload/src/fields/hooks/beforeChange/index.ts:60:11) at processTicksAndRejections (node:internal/process/task_queues:95:5) at updateByID (/Users/tim/repos/payload-reproduce-error/node_modules/.pnpm/payload@2.4.0_typescript@4.8.4_webpack@5.89.0/node_modules/payload/src/collections/operations/updateByID.ts:230:18) "data": [ { "field": "foods", "message": "This relationship field has the following invalid relationships: [object Object] 0" } ], "isOperational": true, "isPublic": false, "status": 400, "name": "ValidationError" }

I was able to recreate your issue. To solve it, you should pass the ID's of the foods, you can by fetching stores like so:

const store: Store = await req.payload.findByID({
  collection: "stores",
  id: storeID,
  depth: 0
});

or you could filter the foods down to just their ID's and then spreading the ID's when merging. You would also likely want to ensure the ID of the food you are editing does not already exist in the store foods list.

[fr3fou]

Still reproducible when running "@payloadcms/db-postgres": "^0.7.0" & "payload": "^2.11.1". Trying to run an afterChange hook to create an object from another (non-global) collection which has a reference to the first collection and get the following error:

error: insert or update on table "test_rels" violates foreign key constraint "tests_rels_users_id_users_id_fk"

[ChillerPerser]

Had the nearly same problem as described. But for me the request to update another collection never finished - no error, not response...nothing. Switched from postgres to mongodb and the error was gone.

[lorenzowijtman]

(Using postgres as well) We've found that if you don't await the update/create call in the afterChange hook, it seems to work as intended. However, this could indicate a timing issue and might still result in an error if the timing is slightly off between the two operations. This is also not a full fix if you need the result for further operations of course

[wkentdag]

I've run into the same problems intermittently. The behavior seems to alternate between the ValidationError, or the pg constraint error. I was able to fix it with two updates:

1. Pass the req as an option to payload operations within the hook
2. Cast doc.id to a number before passing it as value back to payload (if you're using postgres)

const hook: CollectionAfterChangeHook  = async ({ doc, collection, req }) => {
  const data = await req.payload.create({
    collection: 'pages',
    data: {
      post: { 
+       value: Number(doc.id), 
-       value: doc.id,
        relationTo: collection.slug 
      }
    }
+   req,
  })

  return doc
}

Passing req threads the request into the same transaction as the initial change operation, fixing the race condition. The option was introduced in #5068 and I already found a related regression in the search plugin -- I bet this subtly broke a number of hooks.

As for casting the id to a number... there must be some edge case in core where it's sometimes returned as a string. I gave up trying to find the source of that one. Might be related to #5794.

"@payloadcms/db-postgres": "^0.7.1",
"payload": "^2.12.1"

[AkashKhattri]

(Using postgres as well) We've found that if you don't await the update/create call in the afterChange hook, it seems to work as intended. However, this could indicate a timing issue and might still result in an error if the timing is slightly off between the two operations. This is also not a full fix if you need the result for further operations of course

This seems to work. Using postgres as well

[alvara]

(Using postgres as well) We've found that if you don't await the update/create call in the afterChange hook, it seems to work as intended. However, this could indicate a timing issue and might still result in an error if the timing is slightly off between the two operations. This is also not a full fix if you need the result for further operations of course

Thanks for this! was having the same issue as everyone else here. got it working by removing await, but yeah this does pose an issue if we need to do something more complicated..

To share details on my side: Im using Postgres, payload 2.30.1 with db-postgres 0.8.9.

In my debugger, I noticed the expected new record does not show at all in the database when I put a breakpoint in both the afterChange and afterOperation hooks, and get the following error (which makes sense given that the record does not yet exist at this point):

Error setting up initial community organizer: error: insert or update on table "community_organizers_rels" violates foreign key constraint "community_organizers_rels_communities_fk"