drizzle-team / drizzle-orm

Headless TypeScript ORM with a head. Runs on Node, Bun and Deno. Lives on the Edge and yes, it's a JavaScript ORM too 😅
https://orm.drizzle.team
Apache License 2.0
24.8k stars 658 forks source link

[BUG]: Neon transactions not working in Node v20 #2200

Open jakeleventhal opened 7 months ago

jakeleventhal commented 7 months ago

What version of drizzle-orm are you using?

0.30.8

What version of drizzle-kit are you using?

0.20.17

Describe the Bug

I previously had no issues until I tried to upgrade from node 18.19.1 to 20.12.2. When making a parallel insert in a transaction, it seems to lose sight of the transaction scope when using neon serverless.

Below is the db object I am exporting. When I hardcode to always use the postgres-js version, it works fine. If I hardcode to use the neon-serverless version, it fails.

import { neonConfig, Pool } from '@neondatabase/serverless';
import { drizzle as neonDrizzle, NeonDatabase } from 'drizzle-orm/neon-serverless';
import { drizzle } from 'drizzle-orm/postgres-js';
import postgres from 'postgres';
import ws from 'ws';

import * as schema from './schema';

neonConfig.webSocketConstructor = ws;

const drizzleClient = (datasourceUrl: string | undefined) => {
  const connectionString = datasourceUrl || process.env.DATABASE_URL;

  const client = () => {
    if (process.env.SERVERLESS_DRIVER) {
      const pool = new Pool({ connectionString });
      return neonDrizzle(pool, { schema });
    }

    return drizzle(postgres(connectionString!), { schema }) as unknown as NeonDatabase<typeof schema>;
  };

  if (!connectionString) {
    return null as any as ReturnType<typeof client>;
  }

  return client();
};

declare global {
  // eslint-disable-next-line vars-on-top, no-var
  var localDrizzle: ReturnType<typeof drizzleClient>;
}

export const getDrizzleClient = (url?: string): ReturnType<typeof drizzleClient> => {
  if (process.env.SERVER || url) {
    return drizzleClient(url);
  }

  if (!global.localDrizzle) {
    global.localDrizzle = drizzleClient(url);
  }
  return global.localDrizzle;
};

const db = getDrizzleClient();
export default db;

export * from './appData';
export * from './schema';
export * from './types';

Below is my application code

const user = await db.transaction(async (tx) => {
  const [newUser] = await tx
    .insert(users)
    .values({
      email,
      emailVerificationEmail: email,
      id: userId,
      password: Utils.getHash(password),
      stripeCustomerId: await StripeUtils.createStripeCustomer(userId, email),
      ...(!process.env.SERVER && {
        emailVerificationEmail: null,
        emailVerificationId: null
      })
    })
    .returning();

  const [{ id: addressId }] = await tx.insert(addresses).values({ userId }).returning();

  const brandingId = crypto.randomUUID();
  await Promise.all([
    tx.insert(brandings).values({ id: brandingId, userId }).execute(),
    tx.insert(userSettings).values({ addressId, userId }).execute()
  ]);
  await tx.insert(packageInserts).values({ brandingId }).execute();

  return newUser;
});

If I remove the Promise.all and instead do each of the inserts one at a time, I do not get an error. Below is the error I end up getting:

Apr 23, 4:18:05PM error: error: insert or update on table "UserSettings" violates foreign key constraint "UserSettings_userId_User_id_fk"
    at eval (webpack-internal:///(api)/../../../node_modules/.pnpm/@neondatabase+serverless@0.9.0/node_modules/@neondatabase/serverless/index.mjs:1355:74)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 1)
    at async eval (webpack-internal:///(api)/./src/pages/api/account/sign-up.tsx:63:9)
    at async NeonSession.transaction (file:///Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/drizzle-orm@0.30.8_@neondatabase+serverless@0.9.0_@types+pg@8.6.6_@types+react@18.2.68_@verce_bkgk4kkqtyvt7agio77yci4idy/node_modules/drizzle-orm/neon-serverless/session.js:96:22)
    at async __WEBPACK_DEFAULT_EXPORT__.requireUserAuth (webpack-internal:///(api)/./src/pages/api/account/sign-up.tsx:42:18)
    at async eval (webpack-internal:///(api)/../../../packages/riptech/utils/src/errors/client/clientErrorWrapper.ts:50:13)
    at async K (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/compiled/next-server/pages-api.runtime.dev.js:21:2871)
    at async U.render (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/compiled/next-server/pages-api.runtime.dev.js:21:3955)
    at async DevServer.runApi (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/server/next-server.js:600:9)
    at async NextNodeServer.handleCatchallRenderRequest (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/server/next-server.js:269:37)
    at async DevServer.handleRequestImpl (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/server/base-server.js:816:17)
    at async /Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/server/dev/next-dev-server.js:339:20
    at async Span.traceAsyncFn (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/trace/trace.js:154:20)
    at async DevServer.handleRequest (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/next@14.2.2_@babel+core@7.23.3_babel-plugin-macros@3.1.0_react-dom@18.2.0_react@18.2.0__react@18.2.0_sass@1.72.0/node_modules/next/dist/server/dev/next-dev-server.js:336:24) 

Schema shown below for extra info:

export const users = pgTable(
  'User',
  {
    company: text('company').default('').notNull(),
    email: text('email').notNull(),
    emailVerificationEmail: text('emailVerificationEmail'),
    emailVerificationId: text('emailVerificationId').$defaultFn(() => sql`gen_random_uuid()::text`),
    firstName: text('firstName').default('').notNull(),
    id: text('id')
      .primaryKey()
      .$defaultFn(() => sql`gen_random_uuid()::text`)
      .notNull(),
    joinedDate: customTimestampWithTimezone('joinedDate')
      .default(sql`now()`)
      .notNull(),
    lastLoginDate: customTimestampWithTimezone('lastLoginDate')
      .default(sql`now()`)
      .notNull(),
    lastName: text('lastName').default('').notNull(),
    logins: integer('logins').default(1).notNull(),
    password: text('password').notNull(),
    passwordResetId: text('passwordResetId'),
    paymentFailureDate: customTimestampWithTimezone('paymentFailureDate'),
    phone: text('phone').default('').notNull(),
    sampleCreditBalance: doublePrecision('sampleCreditBalance').default(100).notNull(),
    stripeCustomerId: text('stripeCustomerId').notNull()
  },
  (table) => ({
    emailKey: uniqueIndex('User_email_key').on(table.email),
    stripeCustomerIdKey: uniqueIndex('User_stripeCustomerId_key').on(table.stripeCustomerId)
  })
);

export const userSettings = pgTable(
  'UserSettings',
  {
    addressId: text('addressId')
      .notNull()
      .references(() => addresses.id, { onDelete: 'restrict', onUpdate: 'cascade' }),
    canadianBusinessNumber: text('canadianBusinessNumber').default('').notNull(),
    displayCurrency: DisplayCurrencyEnum('displayCurrency').default('USD').notNull(),
    id: text('id')
      .primaryKey()
      .$defaultFn(() => sql`gen_random_uuid()::text`)
      .notNull(),
    // The ids of the user's invalid cards that failed to process payments.
    invalidCardIds: text('invalidCardIds')
      .array()
      .default(sql`'{}'`) // Empty array
      .notNull(),
    onboardingStep: OnboardingStepEnum('onboardingStep').default('PersonalDetails'),
    primaryCardId: text('primaryCardId'),
    themeMode: ThemeModeEnum('themeMode').default('light').notNull(),
    userId: text('userId')
      .notNull()
      .references(() => users.id, { onDelete: 'cascade', onUpdate: 'cascade' })
  },
  (table) => ({
    addressIdKey: uniqueIndex('UserSettings_addressId_key').on(table.addressId),
    userIdKey: uniqueIndex('UserSettings_userId_key').on(table.userId)
  })
);

Expected behavior

It works

Environment & setup

See above

jakeleventhal commented 7 months ago

Cross referencing: https://github.com/neondatabase/serverless/issues/76

jakeleventhal commented 7 months ago

Confirmed this actually wasn't node 20. The real problem is in pnpm it seems https://github.com/pnpm/pnpm/issues/8039