Closed maneike closed 2 months ago
This might be related to https://github.com/payloadcms/payload/issues/5473
I am investigating this.
I think this problem is coming from drizzle-kit. I'm waiting on answers from their team.
@DanRibbens
I've read that creating anyTable
in public
schema would fix the Drizzle issue. While it didn't fix it, now it tries to push the tables into the schema, however it's actually pushing to public
, even after specifying custom schema in both Payload and Drizzle configs. Then after going through all the db push prompts successfully:
+ users table will be created
+ payload_preferences table will be created
+ payload_preferences_rels table will be created
+ payload_migrations table will be created
--- all table conflicts resolved ---
I'm getting the same error:
⨯ Internal error: error: schema "payload" does not exist
at /node_modules/pg-pool/index.js:45:11
...
These are the queries I'm passing to be executed in db.execute(import_drizzle_orm9.sql.raw(query));
and they're being logged just before execution:
// node_modules/drizzle-kit/payload.js
DrizzleORMPgClient = class extends DrizzleDbClient {
async query(query, values) {
console.log("console.log(query):", query)
const res = await this.db.execute(import_drizzle_orm9.sql.raw(query));
return res.rows;
}
async run(query) {
console.log("I'm not being logged out due to error. :(")
const res = await this.db.execute(import_drizzle_orm9.sql.raw(query));
return res.rows;
}
};
Output:
console.log(query): select count(*) as count from "anyTable"
# no CREATE SCHEMA here
console.log(query): CREATE TABLE IF NOT EXISTS "payload"."users" (
"id" serial PRIMARY KEY NOT NULL,
"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
"email" varchar NOT NULL,
"reset_password_token" varchar,
"reset_password_expiration" timestamp(3) with time zone,
"salt" varchar,
"hash" varchar,
"login_attempts" numeric,
"lock_until" timestamp(3) with time zone
);
⨯ Internal error: error: schema "payload" does not exist
at /node_modules/pg-pool/index.js:45:11
...
So it looks like Drizzle is pointing to public
first and only then looking for other schemas and eventually not finding them.
I think this problem is coming from drizzle-kit. I'm waiting on answers from their team.
Probably not, I have used drizzle and drizzle-kit with non-default schemas (not the public
schema) in numerous other projects without payload. It is either a payload issue, or an issue with how payload is using drizzle.
I can share a few more helpful pieces of information:
CREATE SCHEMA
statement.Note that the migration file generated below was created with pnpm run payload migrate:create
using the following adapter config, AND that I have truncated the create statements since there are a lot:
db: postgresAdapter({
pool: {
connectionString: process.env.DATABASE_URI || '',
},
schemaName: 'template_payload_nfc_app'
}),
import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
export async function up({ payload }: MigrateUpArgs): Promise<void> {
await payload.db.drizzle.execute(sql`
CREATE TABLE IF NOT EXISTS "template_payload_nfc_app"."nfc_tags" (
"id" serial PRIMARY KEY NOT NULL,
"xuid" varchar NOT NULL,
"active" boolean NOT NULL,
"tag_name" varchar,
"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL
);
CREATE TABLE IF NOT EXISTS "template_payload_nfc_app"."nfc_tags_rels" (
"id" serial PRIMARY KEY NOT NULL,
"order" integer,
"parent_id" integer NOT NULL,
"path" varchar NOT NULL,
"media_gallery_id" integer
);
CREATE TABLE IF NOT EXISTS "template_payload_nfc_app"."users" (
"id" serial PRIMARY KEY NOT NULL,
"admin" boolean,
"name" varchar,
"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
"email" varchar NOT NULL,
"reset_password_token" varchar,
"reset_password_expiration" timestamp(3) with time zone,
"salt" varchar,
"hash" varchar,
"login_attempts" numeric,
"lock_until" timestamp(3) with time zone
);
`);
};
export async function down({ payload }: MigrateDownArgs): Promise<void> {
await payload.db.drizzle.execute(sql`
DROP TABLE "template_payload_nfc_app"."nfc_tags";
DROP TABLE "template_payload_nfc_app"."nfc_tags_rels";
DROP TABLE "template_payload_nfc_app"."users";
DROP TABLE "template_payload_nfc_app"."media_gallery_media_items";
DROP TABLE "template_payload_nfc_app"."media_gallery";
DROP TABLE "template_payload_nfc_app"."media_gallery_rels";
DROP TABLE "template_payload_nfc_app"."uploads";
DROP TABLE "template_payload_nfc_app"."payload_preferences";
DROP TABLE "template_payload_nfc_app"."payload_preferences_rels";
DROP TABLE "template_payload_nfc_app"."payload_migrations";`);
};
schemas
fieldOnce again, I have not pasted the entire file - just the end of it
{
"id": "3af332c8-094a-443d-a20e-410d184c2bbb",
"prevId": "00000000-0000-0000-0000-000000000000",
"version": "5",
"dialect": "pg",
"tables": {...}, // tables are here, correctly. omitted for illustration & brevity
"enums": {},
"schemas": {},
"_meta": {
"schemas": {}, // there should be stuff here for a schema, but there is not.
"tables": {},
"columns": {}
}
}
still doesn't respect schemaName
db: postgresAdapter({
pool: {
connectionString: process.env.DATABASE_URI || '',
},
schemaName: "payload",
}),
⨯ unhandledRejection: error: there is no parameter $1
at eval (webpack-internal:///(rsc)/./node_modules/pg-pool/index.js:45:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async DrizzleORMPgClient.query (/home/alsherif/New_Dev/dr-brand-backend-storefront/node_modules/drizzle-kit/payload.js:34498:21)
at async /home/alsherif/New_Dev/dr-brand-backend-storefront/node_modules/drizzle-kit/payload.js:2259:46 {
length: 95,
severity: 'ERROR',
code: '42P02',
detail: undefined,
hint: undefined,
position: '175',
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'parse_expr.c',
line: '842',
routine: 'transformParamRef'
}
worked when i changed to the postgres databas and i created a new schema
The problem right now with schema name is that if you have other tables with the same name in other schemas they will conflict. It only works if each schema is using uniquely named table and enums. We're waiting on a fix from drizzle.
@DanRibbens do you know if there is any update for this issue? I'm experiencing the same issue and it would be nice to be able to add payload with an existing project to accelerate the content creation via the CMS.
I have also encountered this bug. @DanRibbens might be right. This obviously prevents any attempts at trying to make two separate Payload CMS instances with collections with duplicate names access the same database, which is what I was trying to do. If for example you have a schema "public" and "test", both with the collection "users", only the Payload CMS instance that tries to access the "public" schema will work, and the other will throw something like the following when attempting to run it locally or anywhere else:
⨯ Internal error: error: relation "users" already exists in schema "test"
I guess this could be resolved if there was a way to add a prefix/suffix to all tables that payload accesses inside the postgresAdapter options, but that would be quite an ugly hack.
@hansfuchs I believe there is already a fix in progress: drizzle-team/drizzle-orm#2363 - Fix RQB behavior for tables with same names in different schemas
I pinged our drizzle friends about this, waiting on a fix.
any update on this? I'd love to use Payload as CMS for our customers but this is kind of a deal breaker...
This was fixed here: https://github.com/payloadcms/payload/pull/7453
This issue has been automatically locked. Please open a new issue if this issue persists with any additional detail.
Link to reproduction
No response
Describe the Bug
I've encountered a bug using payload-3.0-demo, where upon initializing Payload, a custom schema in my database is not being created. I've implemented it like so:
Here is the error after running simple
pnpm dev
:So I've created a schema with the specified name in my database and tables were properly created within it. I have the impression that this is a workaround for the problem, as when modifying the collection (e.g., changing the slug name in pages collection), I've received a similar error.
slug: "title" -> slug: "page_title"
To Reproduce
schemaName: "payload"
topayload.config.ts
pnpm dev
and go tolocalhost:3000/admin
Payload Version
3.0.0-beta.6
Adapters and Plugins
@payloadcms/db-postgres