funmusicplace / mirlo

An open source audio distribution and patronage platform
https://mirlo.space/
GNU General Public License v3.0
124 stars 26 forks source link

Prisma errors on `docker compose up` from fresh install #702

Open olive20xx opened 6 months ago

olive20xx commented 6 months ago

Hi there, would love to contribute but I'm running into errors with Docker while trying to get the project set up locally. I'm on a Macbook Pro M2, so I tried the Apple Chip Errors solution just in case, but DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up just gave me a different error.

Full error logs are included at the bottom of this post.

Steps from a fresh install:

(Rosetta 2 is installed, and Use Rosetta for x86_64/amd64 emulation on Apple Silicon is enabled in Docker Desktop.)

  1. git clone git@github.com:funmusicplace/mirlo.git
  2. yarn -v to check I'm on Yarn 4 ✅
  3. cp .env.example .env
  4. yarn install
  5. docker compose up (see error log below)
  6. delete all containers, images, volumes, and try DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up (see error log below)

Hopefully I'm just making a dumb mistake somewhere? Let me know if you need more info

Btw, docker compose watch seems fine but doesn't seem to run anything on localhost:3000? Is that intentional? Sorry, been a minute since I've had to think in Docker

Error Logs

docker compose up

blackbird-background  | PrismaClientInitializationError: Unable to require(`/var/www/api/prisma/__generated__/libquery_engine-linux-arm64-openssl-1.1.x.so.node`).
blackbird-background  | Prisma cannot find the required `libssl` system library in your system. Please install openssl-1.1.x and try again.
blackbird-background  |
blackbird-background  | Details: libssl.so.1.1: cannot open shared object file: No such file or directory
blackbird-background  |     at Object.loadLibrary (/var/www/api/prisma/__generated__/runtime/library.js:111:10219)
blackbird-background  |     at processTicksAndRejections (node:internal/process/task_queues:95:5)
blackbird-background  |     at async wt.loadEngine (/var/www/api/prisma/__generated__/runtime/library.js:112:448)
blackbird-background  |     at async wt.instantiateLibrary (/var/www/api/prisma/__generated__/runtime/library.js:111:12778) {
blackbird-background  |   clientVersion: '5.13.0',
blackbird-background  |   errorCode: undefined
blackbird-background  | }
blackbird-api         | Prisma schema loaded from schema.prisma
blackbird-api         | Datasource "db": PostgreSQL database "nomads", schema "public" at "pgsql:5432"
blackbird-api         |
blackbird-api         | Error: Schema engine error:
blackbird-api         |
blackbird-background  | info: STARTING WORKER QUEUE {"service":"background-queue"}
blackbird-background  | info: Upload Audio worker started {"service":"background-queue"}
blackbird-background  | info: Optimize Image worker started {"service":"background-queue"}
blackbird-background  | info: Generate Album worker started {"service":"background-queue"}
blackbird-background  | /var/www/api/prisma/__generated__/runtime/library.js:111
blackbird-background  | ${t}`,R(n,!0));this.name="RequestError";this.code="P5010"}};w(Et,"RequestError");async function ur(e,r,t=n=>n){let n=r.clientVersion;................[lots more minified gibberish]

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up

[+] Building 4.6s (19/20)                                             docker:desktop-linux
 => [api internal] load build definition from Dockerfile                              0.0s
 => => transferring dockerfile: 1.15kB                                                0.0s
 => [api internal] load metadata for docker.io/library/node:18-bookworm-slim          1.3s
 => [api internal] load metadata for docker.io/jrottenberg/ffmpeg:5.1-ubuntu2204      1.3s
 => [api internal] load .dockerignore                                                 0.0s
 => => transferring context: 149B                                                     0.0s
 => [api ffmpeg 1/1] FROM docker.io/jrottenberg/ffmpeg:5.1-ubuntu2204@sha256:988ad46  0.0s
 => => resolve docker.io/jrottenberg/ffmpeg:5.1-ubuntu2204@sha256:988ad467b37117dae5  0.0s
 => [api base  1/14] FROM docker.io/library/node:18-bookworm-slim@sha256:cbfb3c98309  0.0s
 => [api internal] load build context                                                 0.1s
 => => transferring context: 174.53kB                                                 0.1s
 => CACHED [api base  2/14] COPY --from=ffmpeg / /                                    0.0s
 => CACHED [api base  3/14] RUN apt-get update -qq                                    0.0s
 => CACHED [api base  4/14] RUN apt-get install -qq libavdevice59 >/dev/null          0.0s
 => CACHED [api base  5/14] WORKDIR /var/www/api                                      0.0s
 => CACHED [api base  6/14] RUN corepack enable                                       0.0s
 => CACHED [api base  7/14] RUN corepack prepare yarn@4.1.1 --activate                0.0s
 => CACHED [api base  8/14] COPY package.json yarn.lock .yarnrc.yml .                 0.0s
 => CACHED [api base  9/14] COPY prisma/package.json ./prisma/package.json            0.0s
 => CACHED [api base 10/14] COPY client/package.json ./client/package.json            0.0s
 => CACHED [api base 11/14] RUN --mount=type=cache,target=/root/.yarn,sharing=locked  0.0s
 => CACHED [api base 12/14] COPY prisma ./prisma                                      0.0s
 => ERROR [api base 13/14] RUN yarn prisma:build                                      3.1s
------
 > [api base 13/14] RUN yarn prisma:build:
6.749 Prisma schema loaded from schema.prisma
7.058 assertion failed [block != nullptr]: BasicBlock requested for unrecognized address
3.058 (BuilderBase.h:550 block_for_offset)
------
failed to solve: process "/bin/sh -c yarn prisma:build" did not complete successfully: exit code: 1
fennifith commented 6 months ago

I PR'd some changes to the docker files recently, so it's likely something related. I've found an issue open in Prisma that looks similar, although it's described as intermittent.

docker compose watch is likely running into the same error as your second snippet, but hidden in the logs - I imagine running docker compose logs -f alongside it would show the same error.

Some things that might be helpful to try:


If all else fails, it should be possible to just run the API outside of the container by setting the following in .env:

...and then docker compose up -d redis minio pgsql, yarn prisma:migrate, and yarn api:watch should get you running.

olive20xx commented 6 months ago

@fennifith Hey, thanks for taking a look. I’ll try your suggestions and let you know what happens.

olive20xx commented 6 months ago

Rebuild the images with DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose build --no-cache to make sure this isn't a caching problem, then try DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up again

Yeah, getting the same error with this, whether or not lines 1 and 6 are commented out.

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose build --no-cache

[+] Building 127.8s (19/20)                                                                                    docker:desktop-linux
 => [api internal] load build definition from Dockerfile                                                                       0.0s
 => => transferring dockerfile: 1.15kB                                                                                         0.0s
 => [api internal] load metadata for docker.io/library/node:18-bookworm-slim                                                   0.6s
 => [api internal] load metadata for docker.io/jrottenberg/ffmpeg:5.1-ubuntu2204                                               0.4s
 => [api internal] load .dockerignore                                                                                          0.0s
 => => transferring context: 149B                                                                                              0.0s
 => CACHED [api ffmpeg 1/1] FROM docker.io/jrottenberg/ffmpeg:5.1-ubuntu2204@sha256:988ad467b37117dae54564a36b4074d403f798f87  0.0s
 => => resolve docker.io/jrottenberg/ffmpeg:5.1-ubuntu2204@sha256:988ad467b37117dae54564a36b4074d403f798f8718c1958d85cb0dc869  0.0s
 => [api internal] load build context                                                                                          0.1s
 => => transferring context: 158.64kB                                                                                          0.1s
 => CACHED [api base  1/14] FROM docker.io/library/node:18-bookworm-slim@sha256:cbfb3c9830932b7b1c2738abf47c66568fc7b06cf782d  0.0s
 => [api base  2/14] COPY --from=ffmpeg / /                                                                                    0.9s
 => [api base  3/14] RUN apt-get update -qq                                                                                    6.6s
 => [api base  4/14] RUN apt-get install -qq libavdevice59 >/dev/null                                                         62.8s
 => [api base  5/14] WORKDIR /var/www/api                                                                                      0.0s
 => [api base  6/14] RUN corepack enable                                                                                       0.5s
 => [api base  7/14] RUN corepack prepare yarn@4.1.1 --activate                                                                1.2s
 => [api base  8/14] COPY package.json yarn.lock .yarnrc.yml .                                                                 0.0s
 => [api base  9/14] COPY prisma/package.json ./prisma/package.json                                                            0.0s
 => [api base 10/14] COPY client/package.json ./client/package.json                                                            0.0s
 => [api base 11/14] RUN --mount=type=cache,target=/root/.yarn,sharing=locked YARN_CACHE_FOLDER=/root/.yarn yarn install --i  51.9s
 => [api base 12/14] COPY prisma ./prisma                                                                                      0.1s
 => ERROR [api base 13/14] RUN yarn prisma:build                                                                               3.1s
------
 > [api base 13/14] RUN yarn prisma:build:
2.750 Prisma schema loaded from schema.prisma
3.071 assertion failed [block != nullptr]: BasicBlock requested for unrecognized address
3.071 (BuilderBase.h:550 block_for_offset)
------
failed to solve: process "/bin/sh -c yarn prisma:build" did not complete successfully: exit code: 1

This needs to happen when the container is running, but before it crashes - might be easier to replace the CMD with a sleep 1000 for this

It's failing on RUN yarn prisma:build, so it doesn't even get to the CMD

If I just run docker compose up, I'm seeing this for prisma version (but, not sure if this would be different with DOCKER_DEFAULT_PLATFORM=linux/amd64)

docker compose exec api yarn prisma version
Environment variables loaded from .env
prisma                  : 5.13.0
@prisma/client          : 5.13.0
Computed binaryTarget   : linux-arm64-openssl-3.0.x
Operating System        : linux
Architecture            : arm64
Node.js                 : v18.20.2
Query Engine (Node-API) : libquery-engine b9a39a7ee606c28e3455d0fd60e78c3ba82b1a2b (at node_modules/@prisma/engines/libquery_engine-linux-arm64-openssl-3.0.x.so.node)
Schema Engine           : schema-engine-cli b9a39a7ee606c28e3455d0fd60e78c3ba82b1a2b (at node_modules/@prisma/engines/schema-engine-linux-arm64-openssl-3.0.x)
Schema Wasm             : @prisma/prisma-schema-wasm 5.13.0-23.b9a39a7ee606c28e3455d0fd60e78c3ba82b1a2b
Default Engines Hash    : b9a39a7ee606c28e3455d0fd60e78c3ba82b1a2b
Studio                  : 0.500.0
Preview Features        : fullTextSearch
olive20xx commented 6 months ago

If all else fails, it should be possible to just run the API outside of the container by setting the following in .env:

  • DATABASE_URL="postgresql://nomads:nomads@localhost:5434/nomads?schema=public"
  • REDIS_HOST=localhost
  • MINIO_HOST=localhost

...and then docker compose up -d redis minio pgsql, yarn prisma:migrate, and yarn api:watch should get you running.

Struggling on this part too. yarn prisma:migrate wasn't finding the DATABASE_URL env

yarn prisma:migrate
Prisma schema loaded from schema.prisma
Datasource "db": PostgreSQL database

Error: Prisma schema validation - (get-config wasm)
Error code: P1012
error: Environment variable not found: DATABASE_URL.
  -->  schema.prisma:10
   |
 9 |   provider = "postgresql"
10 |   url      = env("DATABASE_URL")
   |

Validation Error Count: 1
[Context: getConfig]

Prisma CLI Version : 5.13.0

So I tried npx prisma generate first. Didn't help.

I noticed DATABASE_URL="postgresql://nomads:nomads@localhost:5434/nomads?schema=public" isn't quite accurate to my local setup, so I changed 5434 -> 5432 and put my username/pw instead of nomads:nomads. Still same error.

But when I tried regular yarn prisma migrate dev, it seems to work. All the migrations got applied and it ran seed.ts. Now I see the nomads db in my local Postgres.

Though, even after all that, I can't actually login on the dev environment. I get a 404 error: fetchWrapper.ts:19 POST http://localhost:8080/auth/login 404 (Not Found)

Sidenote: It's only supposed to seed users? Not artists?

fennifith commented 6 months ago

Seeing http://localhost:8080/auth/login makes me wonder if client/.env is missing - could you check if you've copied that from the .env.example in that folder yet? The VITE_API_DOMAIN=http://localhost:3000 should tell it to use http://localhost:3000/auth/login, which should be where the API server is running if you're using the default ports.

This should match the API_DOMAIN='http://localhost:3000' and PORT=3000 variables in the .env file at the project root - and yarn api:watch should tell you 🚀 Server ready at: http://localhost:3000 when it starts.

It's also noteworthy that visiting 127.0.0.1:8080 instead of localhost:8080 in my browser gives me CORS errors for the API - so that's something to look out for. I'm not sure if there's a good workaround for that.

As for yarn prisma:migrate - that might be expecting its .env file to be in the prisma/ dir instead of the root. I'll try to get that fixed in a PR - there are a couple things I'd like to improve to make it easier to set up outside of docker as well. I appreciate all the troubleshooting!

olive20xx commented 6 months ago

Oops, egg on my face 😅

Yeah, I hadn't copied the client/.env. Now I can log in with the admin@admin.com seeded user. 👍🏻

As for yarn prisma:migrate - that might be expecting its .env file to be in the prisma/ dir instead of the root.

Yep! I just tried it with the .env inside prisma/ and it worked.

simonv3 commented 6 months ago

https://github.com/funmusicplace/mirlo/pull/737/files#diff-2ffbfd088aef1953e075d339f49443d39d524af968108f3cb4be7bdde42a7da0

We could add the 127.0.0.1 route to the client seeding script (and if we don't get to that @craigostrin could add it in the DB as a workaround for now)

olive20xx commented 1 month ago

I tried @Jackie's fix from Discord and it worked!

A better way to solve the permissions issue with the Postgres volume is to use a named volume instead of a bind mount.

In docker-compose.api.yml, for the pgsql service, change the volume mapping to read volumes:

  • pgsql_data:/var/lib/postgresql/data
  • pgsql_backups:/backups

and at the bottom of the file, remember to define the new named volumes volumes: minio_storage: {} pgsql_data: {} pgsql_backups: {}

Then I re-started with DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose watch and everything seems to be working.

I'm not experienced enough with Docker to say if this should be a general fix or just a README update.

simonv3 commented 1 month ago

@olive20xx do you want to try putting in a branch & pr and I'll pull it down and see what it does?

I think the DOCKER_DEFAULT_PLATFORM thing would be a readme thing, under the #troubleshooting on mac section