search

lucasthevenet / pglite-utils

24 stars 2 forks source link

First testing feedback #8

Open BracketJohn opened 4 months ago

BracketJohn commented 4 months ago

Hey @lucasthevenet 👋

First of all, again, thanks for this very nice push for a community driven pglite adapter!

I started to test this for our production-app (>70k LOC, >2.4k LOC prisma schema) and am collecting some feedback here, as requested by you here.

Points I noticed:

Technical info: We are running a Nuxt 3 app. The adapter + prisma client is created on startup of the nitro server we are using.

So at this point I'm sadly stuck with testing it, as the server does not start up correctly, or rather: The first prisma query we run (creating a default user, see logs below) fails.

Thanks again though for creating this great adapter & putting your time into it. I'm happy to test & use this further, once it makes sense (note: I'm not assuming free support / resolution from your side, just trying to give helpful feedback in case you want it).

[1] logs from error on pnpm dev

 ERROR  [nitro] [unhandledRejection] memory access out of bounds

  at wasm://wasm/018421ca:wasm-function[8871]:0x43ece4
  at wasm://wasm/018421ca:wasm-function[9159]:0x459928
  at wasm://wasm/018421ca:wasm-function[9162]:0x459a58
  at wasm://wasm/018421ca:wasm-function[3035]:0x1dd00b
  at wasm://wasm/018421ca:wasm-function[10680]:0x494cf8
  at wasm://wasm/018421ca:wasm-function[3046]:0x1dd324
  at wasm://wasm/018421ca:wasm-function[257]:0x14078
  at wasm://wasm/018421ca:wasm-function[4780]:0x2efd94
  at wasm://wasm/018421ca:wasm-function[11106]:0x49628c
  at wasm://wasm/018421ca:wasm-function[9707]:0x49118b

thread 'tokio-runtime-worker' panicked at query-engine/core/src/response_ir/internal.rs:620:73:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

  Invalid prisma.user.create() invocation in
  server/api/auth/[...].ts:87:1

  → 87 return prisma.user.create(
  called Option::unwrap() on a None value

  This is a non-recoverable error which probably happens when the Prisma Query Engine has a panic.

  https://github.com/prisma/prisma/issues/new?body=Hi+Prisma+Team%21+My+Prisma+Client+just+crashed.+This+is+the+report%3A%0A%23%23+Versions%0A%0A%7C+Name++++++++++++%7C+Version++++++++++++%7C%0A%7C-----------------%7C--------------------%7C%0A%7C+Node++++++++++++%7C+v20.12.2+++++++++++%7C+%0A%7C+OS++++++++++++++%7C+darwin-arm64+++++++%7C%0A%7C+Prisma+Client+++%7C+5.14.0+++++++++++++%7C%0A%7C+Query+Engine++++%7C+e9771e62de70f79a5e1c604a2d7c8e2a0a874b48%7C%0A%7C+Database++++++++%7C+postgresql+++++++++%7C%0A%0A%0A%0A%23%23+Logs%0A%60%60%60%0AibraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+already+starting%2C+this.libraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+already+starting%2C+this.libraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+already+starting%2C+this.libraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+already+starting%2C+this.libraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+already+starting%2C+this.libraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+already+starting%2C+this.libraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+already+starting%2C+this.libraryStarted%3A+false%0Aprisma%3Aclient%3AlibraryEngine+library+started%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22typeName%22%5D%7D%7D%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22name%22%5D%7D%7D%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22name%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22name%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22name%22%5D%7D%7D%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22name%22%5D%7D%7D%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3Arequest_handler+%7B%22name%22%3A%22PrismaClientKnownRequestError%22%2C%22code%22%3A%22P2002%22%2C%22clientVersion%22%3A%225.14.0%22%2C%22meta%22%3A%7B%22target%22%3A%5B%22name%22%5D%7D%7D%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3Arequest_handler+%7B%22clientVersion%22%3A%225.14.0%22%7D%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0Aprisma%3Aclient%3AlibraryEngine+sending+request%2C+this.libraryStarted%3A+true%0A%60%60%60%0A%0A%23%23+Client+Snippet%0A%60%60%60ts%0A%2F%2F+PLEASE+FILL+YOUR+CODE+SNIPPET+HERE%0A%60%60%60%0A%0A%23%23+Schema%0A%60%60%60prisma%0A%2F%2F+PLEASE+ADD+YOUR+SCHEMA+HERE+IF+POSSIBLE%0A%60%60%60%0A%0A%23%23+Prisma+Engine+Query%0A%60%60%60%0A%7B%22X%22%3Atrue%7D%7D%7D%0A%60%60%60%0A&title=called+%60Option%3A%3Aunwrap%28%29%60+on+a+%60None%60+value&template=bug_report.yml

  If you want the Prisma team to look into it, please open the link above 🙏
  To increase the chance of success, please post your schema and a snippet of
  how you used Prisma Client in the issue.
janpio commented 4 months ago
  • from this part of the Readme I'm a bit unsure on how to apply the schema (aka: run prisma db push) - it does not seem to be supported, but if it isn't, how can I apply the schema my app needs to run?

I just researched as well, @lucasthevenet has this nice piece of code to run migrations on the fly: https://github.com/lucasthevenet/pglite-utils/blob/77421105f04b8fe2ab82a282db9542eb0cd9b181/sandbox/src/pglite.ts#L6-L12 It uses the PGlite API directly without Prisma, but runs a migration.sql generated by Prisma. Real-world use cases would probably be a bit more complex.

(Prisma has plans to support migrations in code, so that they can also use driver adapters - but that is not a thing that will happen in the next weeks.)

lucasthevenet commented 4 months ago

Thanks for taking some time to test this! I apologize for the naming errors, I changed the name at the last moment and forgot to change it on some places.

As Jan correctly states, currently there is no way to run migrations through Prisma CLI for PGlite. My idea is to make a wrapper so you can run prisma cli commands targeting a PGlite db (haven't found the time yet).

In the meantime, one workaround you can use is to generate a sql file with npx prisma migrate diff --from-empty --to-schema-datamodel prisma/schema.prisma --script > migrate.sql And then run the generated file through code.

As for the error you're getting, could you please tell me what type of dataDir are you targeting? (memory:// or file dir)

BracketJohn commented 4 months ago

Sure thing & no worries - I've worked on a few open-source things & this kind of stuff always happens in the beginning of projects (:

As Jan correctly states, currently there is no way to run migrations through Prisma CLI for PGlite. My idea is to make a wrapper so you can run prisma cli commands targeting a PGlite db (haven't found the time yet).

I thought so - that makes it a bit hard to use, as you can imagine. How would this command work? Would it also rely on migrations/?

In the meantime, one workaround you can use is to generate a sql file with npx prisma migrate diff --from-empty --to-schema-datamodel prisma/schema.prisma --script > migrate.sql And then run the generated file through code.

Thanks, makes sense!

As for the error you're getting, could you please tell me what type of dataDir are you targeting? (memory:// or file dir)

Yes, I was using:

DATABASE_URL="./prisma/pgdata"

I also ran resolve on the path before passing it to the adapter to ensure that it is reprensented correctly (eg: with \ on Windows machines):

// Taken from https://www.npmjs.com/package/pathe
import { resolve } from 'pathe'

const client = new PGlite(resolve(process.env.DATABASE_URL));
const adapter = new PrismaPGlite(client);
const prisma = new PrismaClient({ adapter });

One additional note, I don't know if this changes anything: In our project we extend the prisma-client using prisma.$extend(...), which entered GA in 4.16.0, see more here

lucasthevenet commented 4 months ago

I tried reproducing your error thinking it was related to nuxt/nitro, but had no success, everything seems to work ok for me.

It would be great if you could create a reproduction so we can point out where the error is happening precisely.

I added an example nuxt project that uses Prisma and PGlite, if you want you can base a reproduction out of that example.

BracketJohn commented 4 months ago

Thanks @lucasthevenet -> I'll check it out and test it further + report back!

samwillis commented 2 months ago

Hey, PGlite maintainer here.

This is really awesome!

the ERROR [nitro] [unhandledRejection] memory access out of bounds that @BracketJohn saw is a bug in the current release of PGlite related to TOAST (the way Postgres handles large value), it's fixed in the alpha we just released based on PG v16.3: https://github.com/electric-sql/pglite/releases/tag/v0.2.0-alpha.0

tobiasdiez commented 2 months ago

Is there any update on how to best update the database tables with the prisma schema?

My use case is to use pglite for local development as a in-memory database, so I would be fine with running this as javascript code instead of a proper cli command. Also in such a scenario, how would one create new migrations (to be applied in the production db) if the schema changes?