thirdweb-dev / engine

The most powerful backend engine for web3 apps.
https://thirdweb.com/engine
Apache License 2.0
152 stars 59 forks source link

Error on start Self-Hosted Engine due PrismaORM #504

Closed thedarkknight197 closed 6 months ago

thedarkknight197 commented 6 months ago

Dockercompose

My docker-compose.yml:

version: '3'

services:
  redis:
    image: redis:latest
    expose:
      - 6379

  rabbitmq:
    image: rabbitmq:3-management
    container_name: rabbitmq
    expose:
      - 5672
      - 15672

  migration:
    container_name: migration
    depends_on:
      - redis
      - rabbitmq
    build:
      context: .
      dockerfile: Dockerfile-migrations
    labels:
      - "traefik.enable=true"

  tenant:
    depends_on:
      - redis
      - rabbitmq
    build:
      context: .
      dockerfile: Dockerfile-tenant
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.tenant.rule=Host(`admin.$MY_DOMAIN`)"
      - "traefik.http.routers.tenant.entrypoints=websecure"
      - "traefik.http.routers.tenant.tls.certresolver=lets-encr"
      - "traefik.http.routers.tenant.priority=10" # Priorità più alta per il tenant admin

  multi_tenant:
    depends_on:
      - redis
      - rabbitmq
    build:
      context: .
      dockerfile: Dockerfile-multi_tenant
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.multi_tenant.rule=HostRegexp(`{subdomain:[a-z]+}.${MY_DOMAIN}`)"
      - "traefik.http.routers.multi_tenant.entrypoints=websecure"
      - "traefik.http.routers.multi_tenant.tls.certresolver=lets-encr"
      - "traefik.http.routers.multi_tenant.priority=1" # Priorità più alta per il tenant admin

  traefik:
    image: "traefik:v2.1"
    hostname: "traefik"
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./traefik.yml:/traefik.yml:ro"
      - "./acme.json:/acme.json"
    labels:
      ## HTTP REDIRECT
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      - "traefik.http.routers.redirect-https.rule=hostregexp(`{host:.+}`)"
      - "traefik.http.routers.redirect-https.entrypoints=web"
      - "traefik.http.routers.redirect-https.middlewares=redirect-to-https"

  postgres:
    image: postgres:13.5
    restart: always
    container_name: db
    ports:
      - 5432:5432
    volumes:
      - ./apps/multi_tenant/.docker/dev/postgres-data:/var/lib/postgresql/data
    env_file: "./apps/multi_tenant/.docker/dev/envs/db.env"

  grafana:
    image: grafana/grafana
    user: '104'
    volumes:
      - ./grafana/data:/var/lib/grafana
      - ./grafana/provisioning/:/etc/grafana/provisioning/
      - ./grafana/configuration/grafana.ini:/etc/grafana/grafana.ini
    env_file:
      - ./grafana/config.monitoring
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.grafana.rule=Host(`grafana.$MY_DOMAIN`)"
      - "traefik.http.routers.grafana.entrypoints=websecure"
      - "traefik.http.routers.grafana.tls.certresolver=lets-encr"
      - "traefik.http.services.grafana.loadbalancer.server.port=3000"

  engine:
    image: thirdweb/engine:v0.0.35
    ports:
      - "3005:3005"
    environment:
      - POSTGRES_CONNECTION_URL=postgresql://user:password@db:5432/engine?schema=public
      - ENCRYPTION_PASSWORD=<password>
      - THIRDWEB_API_SECRET_KEY=<token>
      - ADMIN_WALLET_ADDRESS=<address>
      - ENABLE_HTTPS=true
    depends_on:
      - postgres

Error

The db is a postgres docker on the same docker-compose

first run

yarn run v1.22.19
2024-04-18T19:25:18.898131131Z $ yarn prisma:setup:prod && yarn start:run
2024-04-18T19:25:19.056044416Z $ npx tsx ./dist/scripts/setup-db.js
2024-04-18T19:25:21.176989376Z npm WARN exec The following package was not found and will be installed: tsx@4.7.2
2024-04-18T19:25:24.696281941Z prisma:info Starting a postgresql pool with 25 connections.
2024-04-18T19:25:24.865855299Z $ /app/node_modules/.bin/prisma migrate deploy --schema ./dist/prisma/schema.prisma
2024-04-18T19:25:25.286845068Z Prisma schema loaded from dist/prisma/schema.prisma
2024-04-18T19:25:25.295067642Z Datasource "db": PostgreSQL database "engine", schema "public" at "db:5432"
2024-04-18T19:25:25.343622460Z 
2024-04-18T19:25:25.343656981Z 28 migrations found in prisma/migrations
2024-04-18T19:25:25.343662561Z 
2024-04-18T19:25:25.416183618Z Applying migration `20230913025746_init`
2024-04-18T19:25:25.479150141Z Applying migration `20230913061811_triggers`
2024-04-18T19:25:25.483831850Z Error: P3018
2024-04-18T19:25:25.483859071Z 
2024-04-18T19:25:25.483864651Z A migration failed to apply. New migrations cannot be applied before the error is recovered from. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve
2024-04-18T19:25:25.483868121Z 
2024-04-18T19:25:25.483870141Z Migration name: 20230913061811_triggers
2024-04-18T19:25:25.483872051Z 
2024-04-18T19:25:25.483873901Z Database error code: 42601
2024-04-18T19:25:25.483875731Z 
2024-04-18T19:25:25.483877491Z Database error:
2024-04-18T19:25:25.483879381Z ERROR: syntax error at or near "TRIGGER"
2024-04-18T19:25:25.483881321Z 
2024-04-18T19:25:25.483883101Z Position:
2024-04-18T19:25:25.483884911Z  19   )::text);
2024-04-18T19:25:25.483887121Z  20   RETURN NEW;
2024-04-18T19:25:25.483889062Z  21 END;
2024-04-18T19:25:25.483890982Z  22 $function$;
2024-04-18T19:25:25.483893592Z  23
2024-04-18T19:25:25.483895582Z  24 CREATE OR REPLACE TRIGGER transactions_insert_trigger
2024-04-18T19:25:25.483897602Z 
2024-04-18T19:25:25.483899542Z DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState(E42601), message: "syntax error at or near \"TRIGGER\"", detail: None, hint: None, position: Some(Original(551)), where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("scan.l"), line: Some(1176), routine: Some("scanner_yyerror") }
2024-04-18T19:25:25.483902292Z 
2024-04-18T19:25:25.483904082Z 
2024-04-18T19:25:25.496970958Z error Command failed with exit code 1.
2024-04-18T19:25:25.497007820Z info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
2024-04-18T19:25:25.508531930Z node:internal/errors:865
2024-04-18T19:25:25.508560971Z   const err = new Error(message);
2024-04-18T19:25:25.508566681Z               ^
2024-04-18T19:25:25.508569501Z 
2024-04-18T19:25:25.508571651Z Error: Command failed: yarn prisma migrate deploy --schema ./dist/prisma/schema.prisma
2024-04-18T19:25:25.508573681Z     at __node_internal_genericNodeError (node:internal/errors:865:15)
2024-04-18T19:25:25.508575601Z     at checkExecSyncError (node:child_process:890:11)
2024-04-18T19:25:25.508577471Z     at execSync (node:child_process:962:15)
2024-04-18T19:25:25.508579402Z     at main (file:///app/dist/scripts/setup-db.js:21:9) {
2024-04-18T19:25:25.508581312Z   status: 1,
2024-04-18T19:25:25.508583152Z   signal: null,
2024-04-18T19:25:25.508584962Z   output: [ null, null, null ],
2024-04-18T19:25:25.508586772Z   pid: 128,
2024-04-18T19:25:25.508588612Z   stdout: null,
2024-04-18T19:25:25.508590712Z   stderr: null
2024-04-18T19:25:25.508592542Z }
2024-04-18T19:25:25.508594392Z 
2024-04-18T19:25:25.508596252Z Node.js v18.19.1
2024-04-18T19:25:25.518283009Z npm notice 
2024-04-18T19:25:25.518393232Z npm notice New minor version of npm available! 10.2.4 -> 10.5.2
2024-04-18T19:25:25.518402982Z npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.5.2>
2024-04-18T19:25:25.518405932Z npm notice Run `npm install -g npm@10.5.2` to update!
2024-04-18T19:25:25.518408032Z npm notice 
2024-04-18T19:25:25.534472247Z error Command failed with exit code 1.
2024-04-18T19:25:25.534518239Z info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
2024-04-18T19:25:25.550590962Z error Command failed with exit code 1.
2024-04-18T19:25:25.550637739Z info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

second run

yarn run v1.22.19
2024-04-18T18:26:02.912087582Z $ yarn prisma:setup:prod && yarn start:run
2024-04-18T18:26:03.068464891Z $ npx tsx ./dist/scripts/setup-db.js
2024-04-18T18:26:05.845154103Z npm WARN exec The following package was not found and will be installed: tsx@4.7.2
2024-04-18T18:26:08.379764796Z prisma:info Starting a postgresql pool with 25 connections.
2024-04-18T18:26:08.550881710Z $ /app/node_modules/.bin/prisma migrate deploy --schema ./dist/prisma/schema.prisma
2024-04-18T18:26:08.986245819Z Prisma schema loaded from dist/prisma/schema.prisma
2024-04-18T18:26:08.994446911Z Datasource "db": PostgreSQL database "engine", schema "public" at "db:5432"
2024-04-18T18:26:09.036031070Z 
2024-04-18T18:26:09.036070001Z 31 migrations found in prisma/migrations
2024-04-18T18:26:09.036075641Z 
2024-04-18T18:26:09.064493397Z Error: P3009
2024-04-18T18:26:09.064522958Z 
2024-04-18T18:26:09.064528488Z migrate found failed migrations in the target database, new migrations will not be applied. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve
2024-04-18T18:26:09.064532108Z The `20230913061811_triggers` migration started at 2024-04-18 18:24:13.073938 UTC failed
2024-04-18T18:26:09.064534248Z 
2024-04-18T18:26:09.064536048Z 
2024-04-18T18:26:09.076607772Z error Command failed with exit code 1.
2024-04-18T18:26:09.076638923Z info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
2024-04-18T18:26:09.085788331Z node:internal/errors:865
2024-04-18T18:26:09.085811252Z   const err = new Error(message);
2024-04-18T18:26:09.085816812Z               ^
2024-04-18T18:26:09.085819472Z 
2024-04-18T18:26:09.085821502Z Error: Command failed: yarn prisma migrate deploy --schema ./dist/prisma/schema.prisma
2024-04-18T18:26:09.085823462Z     at __node_internal_genericNodeError (node:internal/errors:865:15)
2024-04-18T18:26:09.085825342Z     at checkExecSyncError (node:child_process:890:11)
2024-04-18T18:26:09.085827193Z     at execSync (node:child_process:962:15)
2024-04-18T18:26:09.085829013Z     at main (file:///app/dist/scripts/setup-db.js:21:9) {
2024-04-18T18:26:09.085830903Z   status: 1,
2024-04-18T18:26:09.085832793Z   signal: null,
2024-04-18T18:26:09.085834593Z   output: [ null, null, null ],
2024-04-18T18:26:09.085836493Z   pid: 105,
2024-04-18T18:26:09.085849233Z   stdout: null,
2024-04-18T18:26:09.085851053Z   stderr: null
2024-04-18T18:26:09.085852863Z }
2024-04-18T18:26:09.085854673Z 
2024-04-18T18:26:09.085856503Z Node.js v18.19.1
2024-04-18T18:26:09.095146146Z npm notice 
2024-04-18T18:26:09.095308111Z npm notice New minor version of npm available! 10.2.4 -> 10.5.2
2024-04-18T18:26:09.095316941Z npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.5.2>
2024-04-18T18:26:09.095320041Z npm notice Run `npm install -g npm@10.5.2` to update!
2024-04-18T18:26:09.095322161Z npm notice 
2024-04-18T18:26:09.110663847Z error Command failed with exit code 1.
2024-04-18T18:26:09.110726960Z info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
2024-04-18T18:26:09.126692541Z error Command failed with exit code 1.
2024-04-18T18:26:09.126707171Z info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
warengonzaga commented 6 months ago

Hello @farhanW3, can you take a look into this? Our user here is having trouble with the existing database with Engine.

thedarkknight197 commented 6 months ago

I have updated with full docker-compose.yml

farhanW3 commented 6 months ago

@thedarkknight197

Looks like Postgres version 13 (which you are using) has a different syntax for creating triggers:

postgres:
    image: postgres:13.5

Our trigger creation syntax is, CREATE OR REPLACE ... which is supported >= Postgres 14.

If you can update the Postgres Version and try it out. We will also, make sure to update our docs and reflect this requirement (if not there already)

warengonzaga commented 6 months ago

@thedarkknight197

Looks like Postgres version 13 (which you are using) has a different syntax for creating triggers:

postgres:
    image: postgres:13.5

Our trigger creation syntax is, CREATE OR REPLACE ... which is supported >= Postgres 14.

If you can update the Postgres Version and try it out. We will also, make sure to update our docs and reflect this requirement (if not there already)

I will update our automated script too for deploying locally.

thedarkknight197 commented 6 months ago

@thedarkknight197 Looks like Postgres version 13 (which you are using) has a different syntax for creating triggers:

postgres:
    image: postgres:13.5

Our trigger creation syntax is, CREATE OR REPLACE ... which is supported >= Postgres 14. If you can update the Postgres Version and try it out. We will also, make sure to update our docs and reflect this requirement (if not there already)

I will update our automated script too for deploying locally.

@warengonzaga Can't update postgres 13.5 to 14, on db starting i receive this error:

PostgreSQL Database directory appears to contain a database; Skipping initialization
2024-04-20T12:11:14.066288941Z 
2024-04-20T12:11:14.077212000Z 2024-04-20 12:11:14.077 UTC [1] FATAL:  database files are incompatible with server
2024-04-20T12:11:14.077235099Z 2024-04-20 12:11:14.077 UTC [1] DETAIL:  The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.11 (Debian 14.11-1.pgdg120+2).
farhanW3 commented 6 months ago

Hey @thedarkknight197

looks, like migration is failing and it's because the data directory is not compatible with v15. if you clear the volume attached for postgres, this will go away, but it will also remove everything. Not sure on what data you have on Postgres & whether it's important.

There are multiple ways of migrating, you might have to read up choose what works best for you. One of them is, you can create back-ups of your db & restore it once you have migrated to newer version

But, if you can do a clean install + run, everything, should work smoothly.

let me know.