search

medz / prisma-dart

Prisma Client Dart is an auto-generated type-safe ORM. It uses Prisma Engine as the data access layer and is as consistent as possible with the Prisma Client JS/TS APIs.
https://prisma.pub
BSD 3-Clause "New" or "Revised" License
436 stars 29 forks source link

PrismaInitializationException: in docker container only #233

Closed bohdan1krokhmaliuk closed 1 year ago

bohdan1krokhmaliuk commented 1 year ago

Hello, I am creating simple jwt auth project with dart_frog and prisma (postgresql). Right now I am trying to push all of that into docker container. The thing is that prisma locally works just fine however in container I constantly receive an error and don't know how to fix it.

PrismaInitializationException: Cannot find the query engine binary (Basename: query-engine-darwin-arm64)

docker-compose.yml

version: "3.8"

services:

  api:
    restart: always
    build: ./
    ports:
      - 8080:8080
    environment:
      JWT_ISSUER: my_company_name
      JWT_ACCESS_SECRET: here-goes-your-access-secret
      JWT_REFRESH_SECRET: here-goes-your-refresh-secret
      DATABASE_URL: postgresql://postgres:12345678@db:5432/jwt_auth?schema=public

  db:
    restart: always
    image: postgres:15.3
    ports:
      - 5432:5432
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: jwt_auth
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: 12345678

volumes:
  db_data:

Dockerfile

# An example of using a custom Dockerfile with Dart Frog
# Official Dart image: https://hub.docker.com/_/dart
# Specify the Dart SDK base image version using dart:<version> (ex: dart:2.17)
FROM dart:stable AS build

WORKDIR /app

# Copy Dependencies
COPY packages/db ./packages/db

# Install Dependencies
RUN dart pub get -C packages/db

# Resolve app dependencies.
COPY pubspec.* ./
RUN dart pub get

# Copy app source code and AOT compile it.
COPY . .

# Generate a production build.
RUN dart pub global activate dart_frog_cli
RUN dart pub global run dart_frog_cli:dart_frog build

ARG JWT_ISSUER
ARG JWT_ACCESS_SECRET
ARG JWT_REFRESH_SECRET
ARG DATABASE_URL
# Ensure packages are still up-to-date if anything has changed.
RUN dart pub get --offline
RUN dart compile exe --define=JWT_ISSUER=${JWT_ISSUER} --define=JWT_ACCESS_SECRET=${JWT_ACCESS_SECRET} --define=JWT_REFRESH_SECRET=${JWT_REFRESH_SECRET} --define=DATABASE_URL=${DATABASE_URL} build/bin/server.dart -o build/bin/server

# Build minimal serving image from AOT-compiled `/server` and required system
# libraries and configuration files stored in `/runtime/` from the build stage.
FROM scratch
COPY --from=build /runtime/ /
COPY --from=build /app/build/bin/server /app/bin/
# Uncomment the following line if you are serving static files.
# COPY --from=build /app/build/public /public/

# Start the server.
CMD ["/app/bin/server"]

docker-compose up --build output

jwt_auth-api-1  | ERROR - 2023-06-09 17:32:22.792658
jwt_auth-api-1  | GET /api/v1/user
jwt_auth-api-1  | Error thrown by handler.
jwt_auth-api-1  | PrismaInitializationException: Cannot find the query engine binary (Basename: query-engine-darwin-arm64)
jwt_auth-api-1  | package:orm/binary_engine.dart 79                        BinaryEngine.resolvedExecutable
jwt_auth-api-1  | package:orm/binary_engine.dart 178                       BinaryEngine._createProcess
jwt_auth-api-1  | package:orm/binary_engine.dart 148                       BinaryEngine.start
jwt_auth-api-1  | package:orm/universal_engine.dart 59                     UniversalEngine.request
jwt_auth-api-1  | package:orm/src/client/model_delegate.dart 40            ModelDelegate._execute
jwt_auth-api-1  | package:orm/src/client/model_delegate.dart 29            ModelDelegate.$query
jwt_auth-api-1  | package:db/src/generated/prisma/prisma_client.dart 1244  UserModelDelegateExtension.findUnique.<fn>
jwt_auth-api-1  | package:orm/src/client/prisma_fluent.dart 37             PrismaFluent.queryBuilder.<fn>
jwt_auth-api-1  | package:db/src/generated/prisma/prisma_client.dart 1253  UserModelDelegateExtension.findUnique
jwt_auth-api-1  | app/build/routes/api/v1/user/index.dart 14               onRequest
jwt_auth-api-1  | app/build/bin/server.dart 51                             buildApiV1UserHandler.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 364                    RouterEntry.invoke.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 372                    RouterEntry.invoke
jwt_auth-api-1  | package:dart_frog/src/router.dart 165                    Router.call
jwt_auth-api-1  | package:dart_frog/src/provider.dart 8                    provider.<fn>.<fn>
jwt_auth-api-1  | package:dart_frog/src/provider.dart 8                    provider.<fn>.<fn>
jwt_auth-api-1  | package:dart_frog/src/provider.dart 8                    provider.<fn>.<fn>
jwt_auth-api-1  | app/build/bin/server.dart 36                             buildRootHandler.<fn>
jwt_auth-api-1  | dart:core                                                Function.apply
jwt_auth-api-1  | package:dart_frog/src/router.dart 148                    Router._invokeMountedHandler
jwt_auth-api-1  | package:dart_frog/src/router.dart 105                    Router.mount.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 359                    RouterEntry.invoke.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 372                    RouterEntry.invoke
jwt_auth-api-1  | package:dart_frog/src/router.dart 165                    Router.call
jwt_auth-api-1  | package:dart_frog/src/shelf_adapters.dart 43             toShelfHandler.<fn>
jwt_auth-api-1  | package:shelf/src/cascade.dart 75                        Cascade.handler.<fn>
jwt_auth-api-1  | package:dart_frog/src/shelf_adapters.dart 34             fromShelfHandler.<fn>
jwt_auth-api-1  | package:dart_frog/src/serve.dart 22                      serve.<fn>
jwt_auth-api-1  |
medz commented 1 year ago

@bohdan1krokhmaliuk Hi, See https://github.com/odroe/prisma-dart/blob/main/example/Dockerfile#L32-L42

bohdan1krokhmaliuk commented 1 year ago

Hello @medz, thanks for your response. I adapted my docker file however I still have same problem.

Here is my updated Dockerfile

# An example of using a custom Dockerfile with Dart Frog
# Official Dart image: https://hub.docker.com/_/dart
# Specify the Dart SDK base image version using dart:<version> (ex: dart:2.17)
FROM dart:stable AS build

# Install Node.js LTS to builder
RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - &&\
    apt-get install -y nodejs

WORKDIR /app

# Copy Dependencies
COPY packages/db ./packages/db

# Install Dependencies
RUN dart pub get -C packages/db

# Resolve app dependencies.
COPY pubspec.* ./
RUN dart pub get

# Copy app source code and AOT compile it.
COPY . .

# Install Prisma CLI
RUN npm install prisma

# Generate a production build.
RUN dart pub global activate dart_frog_cli
RUN dart pub global run dart_frog_cli:dart_frog build

ARG JWT_ISSUER
ARG JWT_ACCESS_SECRET
ARG JWT_REFRESH_SECRET
ARG DATABASE_URL
# Ensure packages are still up-to-date if anything has changed.
RUN dart pub get --offline
RUN dart compile exe --define=JWT_ISSUER=${JWT_ISSUER} --define=JWT_ACCESS_SECRET=${JWT_ACCESS_SECRET} --define=JWT_REFRESH_SECRET=${JWT_REFRESH_SECRET} --define=DATABASE_URL=${DATABASE_URL} build/bin/server.dart -o build/bin/server

# Build minimal serving image from AOT-compiled `/server` and required system
# libraries and configuration files stored in `/runtime/` from the build stage.
FROM scratch
COPY --from=build /runtime/ /

# Copy runtime dependencies
COPY --from=odroe/prisma-dart:latest / /

# Copy executable
COPY --from=build /app/build/bin/server /app/bin/
# Uncomment the following line if you are serving static files.
# COPY --from=build /app/build/public /public/

# Copy Prisma Engine
COPY --from=build /app/node_modules/prisma/query-engine-* /

# Start the server.
CMD ["/app/bin/server"]

docker-compose build output

=> [internal] load build definition from Dockerfile                                                                          0.0s
 => => transferring dockerfile: 1.67kB                                                                                        0.0s
 => [internal] load .dockerignore                                                                                             0.0s
 => => transferring context: 2B                                                                                               0.0s
 => [internal] load metadata for docker.io/odroe/prisma-dart:latest                                                           2.4s
 => [internal] load metadata for docker.io/library/dart:stable                                                                2.5s
 => [auth] odroe/prisma-dart:pull token for registry-1.docker.io                                                              0.0s
 => [auth] library/dart:pull token for registry-1.docker.io                                                                   0.0s
 => [internal] load build context                                                                                             0.0s
 => => transferring context: 24.60kB                                                                                          0.0s
 => CACHED [build  1/13] FROM docker.io/library/dart:stable@sha256:52043e9e58cf996b1da23a909cf09c8720e7e50500a0b6d4ebac9fba9  0.0s
 => CACHED FROM docker.io/odroe/prisma-dart:latest@sha256:f6b167d731cde1fd931ea22f5bfbac80910059cefe5d7c4a530c0a9f8676a432    0.0s
 => [build  2/13] RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - &&    apt-get install -y nodejs             10.1s
 => [build  3/13] WORKDIR /app                                                                                                0.0s 
 => [build  4/13] COPY packages/db ./packages/db                                                                              0.1s 
 => [build  5/13] RUN dart pub get -C packages/db                                                                             4.9s 
 => [build  6/13] COPY pubspec.* ./                                                                                           0.0s 
 => [build  7/13] RUN dart pub get                                                                                            3.0s 
 => [build  8/13] COPY . .                                                                                                    0.1s
 => [build  9/13] RUN npm install prisma                                                                                      3.8s
 => [build 10/13] RUN dart pub global activate dart_frog_cli                                                                  6.7s
 => [build 11/13] RUN dart pub global run dart_frog_cli:dart_frog build                                                      11.0s
 => [build 12/13] RUN dart pub get --offline                                                                                  0.8s
 => [build 13/13] RUN dart compile exe --define=JWT_ISSUER=${JWT_ISSUER} --define=JWT_ACCESS_SECRET=${JWT_ACCESS_SECRET} --d  5.8s
 => [stage-1 1/4] COPY --from=build /runtime/ /                                                                               0.0s
 => [stage-1 2/4] COPY --from=odroe/prisma-dart:latest / /                                                                    0.0s
 => [stage-1 3/4] COPY --from=build /app/build/bin/server /app/bin/                                                           0.0s
 => [stage-1 4/4] COPY --from=build /app/node_modules/prisma/query-engine-* /                                                 0.0s
 => exporting to image                                                                                                        0.0s
 => => exporting layers                                                                                                       0.0s
 => => writing image sha256:b9ecdea83b78e91ddfad1da940cfa4c2e7a08ae67391a5851e16bd022ff97fe7                                  0.0s
 => => naming to docker.io/library/jwt_auth-api

And here still same output for docker-compose up with already updated image.

jwt_auth-api-1  | ERROR - 2023-06-12 13:51:56.911030
jwt_auth-api-1  | GET /api/v1/user
jwt_auth-api-1  | Error thrown by handler.
jwt_auth-api-1  | PrismaInitializationException: Cannot find the query engine binary (Basename: query-engine-darwin-arm64)
jwt_auth-api-1  | package:orm/binary_engine.dart 79                        BinaryEngine.resolvedExecutable
jwt_auth-api-1  | package:orm/binary_engine.dart 178                       BinaryEngine._createProcess
jwt_auth-api-1  | package:orm/binary_engine.dart 148                       BinaryEngine.start
jwt_auth-api-1  | package:orm/universal_engine.dart 59                     UniversalEngine.request
jwt_auth-api-1  | package:orm/src/client/model_delegate.dart 40            ModelDelegate._execute
jwt_auth-api-1  | package:orm/src/client/model_delegate.dart 29            ModelDelegate.$query
jwt_auth-api-1  | package:db/src/generated/prisma/prisma_client.dart 1244  UserModelDelegateExtension.findUnique.<fn>
jwt_auth-api-1  | package:orm/src/client/prisma_fluent.dart 37             PrismaFluent.queryBuilder.<fn>
jwt_auth-api-1  | package:db/src/generated/prisma/prisma_client.dart 1253  UserModelDelegateExtension.findUnique
jwt_auth-api-1  | app/build/routes/api/v1/user/index.dart 14               onRequest
jwt_auth-api-1  | app/build/bin/server.dart 51                             buildApiV1UserHandler.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 364                    RouterEntry.invoke.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 372                    RouterEntry.invoke
jwt_auth-api-1  | package:dart_frog/src/router.dart 165                    Router.call
jwt_auth-api-1  | package:dart_frog/src/provider.dart 8                    provider.<fn>.<fn>
jwt_auth-api-1  | package:dart_frog/src/provider.dart 8                    provider.<fn>.<fn>
jwt_auth-api-1  | package:dart_frog/src/provider.dart 8                    provider.<fn>.<fn>
jwt_auth-api-1  | app/build/bin/server.dart 36                             buildRootHandler.<fn>
jwt_auth-api-1  | dart:core                                                Function.apply
jwt_auth-api-1  | package:dart_frog/src/router.dart 148                    Router._invokeMountedHandler
jwt_auth-api-1  | package:dart_frog/src/router.dart 105                    Router.mount.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 359                    RouterEntry.invoke.<fn>
jwt_auth-api-1  | package:dart_frog/src/router.dart 372                    RouterEntry.invoke
jwt_auth-api-1  | package:dart_frog/src/router.dart 165                    Router.call
jwt_auth-api-1  | package:dart_frog/src/shelf_adapters.dart 43             toShelfHandler.<fn>
jwt_auth-api-1  | package:shelf/src/cascade.dart 75                        Cascade.handler.<fn>
jwt_auth-api-1  | package:dart_frog/src/shelf_adapters.dart 34             fromShelfHandler.<fn>
jwt_auth-api-1  | package:dart_frog/src/serve.dart 22                      serve.<fn>
jwt_auth-api-1  |
medz commented 1 year ago

@bohdan1krokhmaliuk One step away from success, adapt COPY --from=build /app/node_modules/prisma/query-engine-* / to COPY --from=build /app/node_modules/prisma/query-engine-* /app/bin/

The reason for this is that the engine looks for the following locations:

  1. PWD (Resolve CMD /app/bin/server) - .
  2. ./query-engine-*
  3. ./node_modules/prisma/query-engine-*
  4. ./.dart_tool/prisma/query-engine-*

So, you should copy the Prisma binary engine to the same directory as your binary program~

bohdan1krokhmaliuk commented 1 year ago

@medz thank you, you showed me direction where to move. Unfortunatelly your solution didn't work (still same problem). I tried all possible options of placing query engine with both executable in app/bin and in root, however none of those worked. So I started debugging.

I executed RUN ls /app/node_modules/prisma in Dockerfile and I received this output showing that there is no engine

#20 0.315 LICENSE
#20 0.315 README.md
#20 0.315 build
#20 0.315 package.json
#20 0.315 preinstall
#20 0.315 prisma-client
#20 0.315 scripts

So I suppose that engine is generated during npx prisma generate which looks not reliable for me, as I should have code dependant on later on generated code.

  1. So @medz is it possible to keep generated code in my repo, and somehow generate engine as well inside of the repo?
  2. Will locally generated engine work in container?
  3. Is engine dependant on schema.prisma file(aka if it is generated once - will old engine work with newly generated dart code)?
  4. Why engine is needed at all? Is it middleware beetween dart and db (aka dart talks to engine, engine talks to db?) Is communication beetween dart and db possible with generated client but without engine?

UPD: checked case where simply copypasted local engine into container - looks like this is not working. This is exception I get

jwt_auth-api-1  | ERROR - 2023-06-13 17:31:12.313609
jwt_auth-api-1  | GET /api/v1/user
jwt_auth-api-1  | Error thrown by handler.
jwt_auth-api-1  | ProcessException: No such file or directory
jwt_auth-api-1  |   Command: /query-engine-darwin-arm64 --enable-raw-queries --enable-metrics --enable-open-telemetry --port 33553 --host 127.0.0.1
medz commented 1 year ago

@bohdan1krokhmaliuk I see your problem, you can add npx prisma generate command in your build container, it will download the engine that matches the current architecture. There is currently no way to skip copying of the binary engine, and the Prisma binary engine is distributed via a different architecture (which is managed by the Prisma CLI).

I'm trying to make a dylib version of the Prisma engine, but progress has been slow as I've been under a lot of personal stress lately.

bohdan1krokhmaliuk commented 1 year ago

@medz thanks! I think issue can be closed :)