drizzle-team / drizzle-orm

Headless TypeScript ORM with a head. Runs on Node, Bun and Deno. Lives on the Edge and yes, it's a JavaScript ORM too 😅
https://orm.drizzle.team
Apache License 2.0
24.1k stars 608 forks source link

[BUG]: Querying relations which contain bigint columns from Pglite causes precision loss #3106

Open opl- opened 2 weeks ago

opl- commented 2 weeks ago

What version of drizzle-orm are you using?

0.33.0

What version of drizzle-kit are you using?

0.24.2

Describe the Bug

Drizzle-ORM queries relations from PostgreSQL as JSON arrays. Pglite uses the following parser when it encounters a JSON data type:

https://github.com/electric-sql/pglite/blob/0c7a62753bc3dfe2813c081e21e9b37cbcf1ab39/packages/pglite/src/types.ts#L118

As such, all the columns of the relation go through JSON.parse, including ones stored as bigint. This results in precision loss for integers outside of the Number.MIN_SAFE_INTEGER..Number.MAX_SAFE_INTEGER range.

While I haven't tested this, this issue might also occur with other data types, like decimal. It might also occur using other Pg drivers.

Reproduction

export const user = pgTable('user', {
    id: bigint('id', { mode: 'bigint' }).primaryKey(),
});

export const userDetails = pgTable('userDetails', {
    userId: bigint('id', { mode: 'bigint' }).primaryKey().references(() => user.id),
    otherColumn: bigint('otherColumn', { mode: 'bigint' }).notNull(),
});

export const userRelations = relations(user, ({ many, one }) => ({
    details: one(userDetails, {
        fields: [user.id],
        references: [userDetails.id],
    }),
}));

const userId = BigInt(Number.MAX_SAFE_INTEGER) + 20n;
const reallyBigInt = BigInt(Number.MAX_SAFE_INTEGER) * 3n;

await drizzle.insert(user).values({
    id: userId,
});

await drizzle.insert(userDetails).values({
    userId,
    otherColumn: reallyBigInt,
}).execute();

const result = await drizzle.query.user.findFirst({
    with: {
        details: true,
    },
}).execute();

assert(result.id === userId); // true
assert(result.details.userId === userId); // false
assert(result.details.otherColumn === reallyBigInt); // false

Expected behavior

BigInts should be parsed as specified in the column type. Ideally, this could be specified as string (#813).

Environment & setup

Node.js v22.8.0

opl- commented 2 weeks ago

Because Pglite allows specifying data type parsers when running a query, that is while calling code which normally gets called by Drizzle ORM, the workaround ends up rather ugly.

The following code converts all numbers contained in JSON into bigints, assuming they fall outside of the safe range for the JavaScript number type, and don't contain a decimal point.

import { types } from '@electric-sql/pglite';
import { PgliteSession } from 'drizzle-orm/pglite';

const jsonParser = (rawJson: string): any => {
    // context argument seems to be unknown to the current types for Node.
    return JSON.parse(rawJson, <any> ((_key: string, value: any, context: { source: string; }) => {
        if (typeof(value) === 'number' && context.source.indexOf('.') === -1) {
            return types.types.bigint.parse(context.source);
        }

        return value;
    }));
};;

const pgliteSessionPrototype = PgliteSession.prototype as any;
pgliteSessionPrototype.prepareQuery_ = pgliteSessionPrototype.prepareQuery;
pgliteSessionPrototype.prepareQuery = function prepareQueryWrapper(...args: any[]) {
    const preparedQuery = pgliteSessionPrototype.prepareQuery_.apply(this, args);

    preparedQuery.queryConfig.parsers[types.JSON] = jsonParser;
    preparedQuery.rawQueryConfig.parsers[types.JSON] = jsonParser;

    return preparedQuery;
};