search

subquery / network-indexer-services

Services that Indexers run to connect and serve data to the SubQuery Network
GNU General Public License v3.0
12 stars 15 forks source link

Kubernetes Compatibility #527

Open pamanseau opened 1 month ago

pamanseau commented 1 month ago

When deploying in Kubernetes the application indexer-coordinator is not connecting to Postgres.

With NODE_ENV=local, the connection works, but then it tries to connect to the docker.sock and terminates. With NODE_ENV=production the error Unable to connect to the database

Tested with either the ENV variables or command arguments

DB_HOST: graph.postgres
DB_NAME: coordinator
DB_USERNAME: postgres
DB_PASSWORD: xxx-xxx-xxx-xxx
DB_PORT: 5432
LOG_LEVEL: debug
NODE_ENV: production
          command: [ "node", "dist/main.js" ]
          args:
            - --network-endpoint=https://mainnet.base.org
            - --network=mainnet
            - --postgres-host=graph.postgres
            - --postgres-database=coordinator
            - --postgres-username=postgres
            - --postgres-password=xxx-xxx-xxx-xxx
            - --postgres-port=5432
            - --postgres-ssl-mode=disabled
            - --port=8000
            - --start-port=3100
            - --secret-key=287sjsd928aD2
            - --ipfs=https://unauthipfs.subquery.network/ipfs/api/v0
pamanseau commented 1 month ago

NODE_ENV=production

graphql-tag enforces all fragment names across your application to be unique; read more about
this in the docs: http://dev.apollodata.com/core/fragments.html#unique-names
2024-10-09T11:11:11.145Z <nestjs> INFO Starting Nest application...
2024-10-09T11:11:11.200Z <nestjs> INFO TypeOrmModule dependencies initialized
2024-10-09T11:11:11.200Z <nestjs> INFO SubscriptionModule dependencies initialized
2024-10-09T11:11:11.200Z <nestjs> INFO DBModule dependencies initialized
2024-10-09T11:11:11.200Z <nestjs> INFO ConfigureModule dependencies initialized
2024-10-09T11:11:11.200Z <nestjs> INFO TypeOrmModule dependencies initialized
2024-10-09T11:11:11.202Z <nestjs> INFO PrometheusModule dependencies initialized
2024-10-09T11:11:11.202Z <nestjs> INFO DiscoveryModule dependencies initialized
2024-10-09T11:11:11.202Z <nestjs> INFO ServeStaticModule dependencies initialized
2024-10-09T11:11:11.202Z <nestjs> INFO NetworkModule dependencies initialized
2024-10-09T11:11:11.202Z <nestjs> INFO MetricsModule dependencies initialized
2024-10-09T11:11:11.202Z <nestjs> INFO ScheduleModule dependencies initialized
2024-10-09T11:11:11.202Z <nestjs> INFO EventEmitterModule dependencies initialized
2024-10-09T11:11:11.203Z <nestjs> INFO GraphQLSchemaBuilderModule dependencies initialized
2024-10-09T11:11:11.203Z <nestjs> INFO GraphQLModule dependencies initialized
2024-10-09T11:11:11.553Z <nestjs> ERROR Unable to connect to the database. Retrying (1)...
2024-10-09T11:11:14.710Z <nestjs> ERROR Unable to connect to the database. Retrying (2)...
2024-10-09T11:11:17.904Z <nestjs> ERROR Unable to connect to the database. Retrying (3)...
2024-10-09T11:11:21.083Z <nestjs> ERROR Unable to connect to the database. Retrying (4)...
2024-10-09T11:11:24.242Z <nestjs> ERROR Unable to connect to the database. Retrying (5)...
2024-10-09T11:11:27.507Z <nestjs> ERROR Unable to connect to the database. Retrying (6)...
2024-10-09T11:11:30.710Z <nestjs> ERROR Unable to connect to the database. Retrying (7)...
2024-10-09T11:11:33.902Z <nestjs> ERROR Unable to connect to the database. Retrying (8)...
2024-10-09T11:11:37.093Z <nestjs> ERROR Unable to connect to the database. Retrying (9)...
2024-10-09T11:11:37.093Z <nestjs> ERROR function uuid_generate_v4() does not exist

NODE_ENV=local

Warning: fragment with name ProjectFields already exists.
graphql-tag enforces all fragment names across your application to be unique; read more about
this in the docs: http://dev.apollodata.com/core/fragments.html#unique-names
2024-10-09T11:21:09.428Z <[95mnestjs[39m> [32mINFO[39m Starting Nest application... 
2024-10-09T11:21:09.485Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.485Z <[95mnestjs[39m> [32mINFO[39m SubscriptionModule dependencies initialized 
2024-10-09T11:21:09.485Z <[95mnestjs[39m> [32mINFO[39m DBModule dependencies initialized 
2024-10-09T11:21:09.485Z <[95mnestjs[39m> [32mINFO[39m ConfigureModule dependencies initialized 
2024-10-09T11:21:09.485Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.486Z <[95mnestjs[39m> [32mINFO[39m PrometheusModule dependencies initialized 
2024-10-09T11:21:09.486Z <[95mnestjs[39m> [32mINFO[39m DiscoveryModule dependencies initialized 
2024-10-09T11:21:09.487Z <[95mnestjs[39m> [32mINFO[39m ServeStaticModule dependencies initialized 
2024-10-09T11:21:09.487Z <[95mnestjs[39m> [32mINFO[39m NetworkModule dependencies initialized 
2024-10-09T11:21:09.487Z <[95mnestjs[39m> [32mINFO[39m MetricsModule dependencies initialized 
2024-10-09T11:21:09.487Z <[95mnestjs[39m> [32mINFO[39m ScheduleModule dependencies initialized 
2024-10-09T11:21:09.487Z <[95mnestjs[39m> [32mINFO[39m EventEmitterModule dependencies initialized 
2024-10-09T11:21:09.487Z <[95mnestjs[39m> [32mINFO[39m GraphQLSchemaBuilderModule dependencies initialized 
2024-10-09T11:21:09.487Z <[95mnestjs[39m> [32mINFO[39m GraphQLModule dependencies initialized 
query: SELECT * FROM current_schema()
query: CREATE EXTENSION IF NOT EXISTS "uuid-ossp"
query: SELECT version();
query: SELECT * FROM "information_schema"."tables" WHERE "table_schema" = 'public' AND "table_name" = 'migrations'
query: SELECT * FROM "migrations" "migrations" ORDER BY "id" DESC
2024-10-09T11:21:09.725Z <[95mnestjs[39m> [32mINFO[39m TypeOrmCoreModule dependencies initialized 
2024-10-09T11:21:09.726Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.726Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.726Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.726Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.726Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.726Z <[95mnestjs[39m> [32mINFO[39m TypeOrmModule dependencies initialized 
2024-10-09T11:21:09.726Z <[95mnestjs[39m> [32mINFO[39m DBModule dependencies initialized 
2024-10-09T11:21:09.790Z <[95mnestjs[39m> [32mINFO[39m ChainModule dependencies initialized 
2024-10-09T11:21:09.791Z <[95mnestjs[39m> [32mINFO[39m ConfigModule dependencies initialized 
2024-10-09T11:21:09.791Z <[95mnestjs[39m> [32mINFO[39m StatsModule dependencies initialized 
2024-10-09T11:21:09.791Z <[95mnestjs[39m> [32mINFO[39m CoreModule dependencies initialized 
2024-10-09T11:21:09.791Z <[95mnestjs[39m> [32mINFO[39m AppModule dependencies initialized 
2024-10-09T11:21:09.795Z <[95mmonitor[39m> [32mINFO[39m check node health started 
2024-10-09T11:21:09.795Z <[95mnestjs[39m> [32mINFO[39m PaygModule dependencies initialized 
2024-10-09T11:21:09.795Z <[95mnestjs[39m> [32mINFO[39m RewardModule dependencies initialized 
2024-10-09T11:21:09.797Z <[95mnestjs[39m> [32mINFO[39m MonitorModule dependencies initialized 
2024-10-09T11:21:09.797Z <[95mnestjs[39m> [32mINFO[39m ProjectModule dependencies initialized 
2024-10-09T11:21:09.800Z <[95mnestjs[39m> [32mINFO[39m AdminController {/}: 
2024-10-09T11:21:09.801Z <[95mnestjs[39m> [32mINFO[39m Mapped {/env.js, GET} route 
2024-10-09T11:21:09.801Z <[95mnestjs[39m> [32mINFO[39m AgreementController {/agreements}: 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m Mapped {/agreements/:id, GET} route 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m MonitorController {/monitor}: 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m Mapped {/monitor, GET} route 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m Mapped {/monitor, POST} route 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m PrometheusController {/metrics}: 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m Mapped {/metrics, GET} route 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m StatsController {/stats}: 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m Mapped {/stats/:deploymentId/:timestamp, POST} route 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m Mapped {/stats, POST} route 
2024-10-09T11:21:09.802Z <[95mnestjs[39m> [32mINFO[39m Mapped {/stats/:deploymentId/:from/:to, GET} route 
query: SELECT "ProjectEntity"."id" AS "ProjectEntity_id", "ProjectEntity"."status" AS "ProjectEntity_status", "ProjectEntity"."chainType" AS "ProjectEntity_chainType", "ProjectEntity"."projectType" AS "ProjectEntity_projectType", "ProjectEntity"."hostType" AS "ProjectEntity_hostType", "ProjectEntity"."rateLimit" AS "ProjectEntity_rateLimit", "ProjectEntity"."nodeEndpoint" AS "ProjectEntity_nodeEndpoint", "ProjectEntity"."queryEndpoint" AS "ProjectEntity_queryEndpoint", "ProjectEntity"."serviceEndpoints" AS "ProjectEntity_serviceEndpoints", "ProjectEntity"."details" AS "ProjectEntity_details", "ProjectEntity"."manifest" AS "ProjectEntity_manifest", "ProjectEntity"."baseConfig" AS "ProjectEntity_baseConfig", "ProjectEntity"."advancedConfig" AS "ProjectEntity_advancedConfig", "ProjectEntity"."projectConfig" AS "ProjectEntity_projectConfig" FROM "project_entity" "ProjectEntity"
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: connect ENOENT /var/run/docker.sock
    at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1278:16) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'connect',
  address: '/var/run/docker.sock'
}
pamanseau commented 1 month ago

Branch #529

pamanseau commented 3 weeks ago

Same error with v2.6.0