electric-sql / electric

Sync little subsets of your Postgres data into local apps and services.
https://electric-sql.com
Apache License 2.0
6.44k stars 155 forks source link

MODULE ERROR: Electric.Replication.PostgresConnector #877

Closed andrezimpel closed 9 months ago

andrezimpel commented 9 months ago

Setting up a new service with Supabase as database on fly.io result sin the following error for me: MODULE ERROR: Electric.Replication.PostgresConnector

2024-01-24T11:13:59.064 app[4d89d944a327d8] ams [info] ▓ ┌────────────────────────────────────────────────────────┐
2024-01-24T11:13:59.064 app[4d89d944a327d8] ams [info] ▓ │ MODULE ERROR: Electric.Replication.PostgresConnector │
2024-01-24T11:13:59.064 app[4d89d944a327d8] ams [info] ▓ ┕━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙
2024-01-24T11:13:59.064 app[4d89d944a327d8] ams [info] ▓
2024-01-24T11:13:59.064 app[4d89d944a327d8] ams [info] ▓ Failed to start child :postgres_producer:
2024-01-24T11:13:59.064 app[4d89d944a327d8] ams [info] ▓ {{:case_clause, {:error, {:error, :error, "42601", :syntax_error, "syntax error at or near \"CREATE_REPLICATION_SLOT\"", [file: "scan.l", line: "1188", position: "1", routine: "scanner_yyerror", severity: "ERROR"]}}}, [{Electric.Replication.Postgres.Client, :create_slot, 2, [file: ~c"lib/electric/replication/postgres/client.ex", line: 142]}, {Electric.Replication.Postgres.LogicalReplicationProducer, :init, 1, [file: ~c"lib/electric/replication/postgres/logical_replication_producer.ex", line: 91]}, {GenStage, :init, 1, [file: ~c"lib/gen_stage.ex", line: 1816]}, {:gen_server, :init_it, 2, [file: ~c"gen_server.erl", line: 851]}, {:gen_server, :init_it, 6, [file: ~c"gen_server.erl", line: 814]}, {:proc_lib, :init_p_do_apply, 3, [file: ~c"proc_lib.erl", line: 240]}]}

This only happens with Supabase.

linear[bot] commented 9 months ago

VAX-1578 MODULE ERROR: Electric.Replication.PostgresConnector

alco commented 9 months ago

Hey @andrezimpel. Can you share your Electric configuration and Postgres logs that correlate in time with the error? Redact any secret credentials from your config but please try to preserve the structure of individual options.

Postgres logs can be found in Supabase by clicking on the Logs button in the sidebar and selecting Postgres in the list on the left. Look for errors that were log at the same time when Electric logged its error.

Screenshot from 2024-01-24 13-45-55

marksteve commented 9 months ago

Hitting the same error. Here's the relevant log metadata for me:

{
  "file": null,
  "host": "db-xxxxxxxxxxxxxxxxxxxx",
  "metadata": [],
  "parsed": [
    {
      "application_name": "Supavisor",
      "backend_type": "client backend",
      "command_tag": "idle",
      "connection_from": "10.103.76.103:55522",
      "context": null,
      "database_name": "postgres",
      "detail": null,
      "error_severity": "ERROR",
      "hint": null,
      "internal_query": null,
      "internal_query_pos": null,
      "leader_pid": null,
      "location": null,
      "process_id": 139183,
      "query": "CREATE_REPLICATION_SLOT \"electric_replication_out_postgres\" LOGICAL pgoutput NOEXPORT_SNAPSHOT",
      "query_id": 0,
      "query_pos": 1,
      "session_id": "65b25f66.21faf",
      "session_line_num": 4,
      "session_start_time": "2024-01-25 13:17:26 UTC",
      "sql_state_code": "42601",
      "timestamp": "2024-01-25 13:17:27.146 UTC",
      "transaction_id": 0,
      "user_name": "postgres",
      "virtual_transaction_id": "6/383037"
    }
  ],
  "parsed_from": null,
  "project": null,
  "source_type": null
}
alco commented 9 months ago

Hey @andrezimpel and @marksteve. I have checked Supabase's database settings and noticed that it now defaults to showing a connection string for the connection pool whereas before it used to show a direct connection string first. Looks like Supabase's connection pool does not support replication connections.

Uncheck the "Use connection pooling" checkbox and copy the resulting URI into your DATABASE_URL configuration.

In the meantime, I've opened https://github.com/electric-sql/electric/pull/896 to improve the error message.

alco commented 9 months ago

Forgot to include the screenshot in my previous comment: Screenshot from 2024-01-26 17-33-11

andrezimpel commented 9 months ago

@alco true, thank you! it seems like supabase now disabled direct access with IPv4. Generating the client now fails for me. :/

$ electric-sql generate
Generating Electric client...
Service URL: http://workplace-sync-service.fly.dev
Proxy URL: postgresql://prisma:********@workplace-sync-service.fly.dev:65432/workplace
Error: Command failed: node xxx/build/index.js db pull --schema=".electric_migrations_tmp_i4bX42/prisma/schema.prisma"
Error: P1017

Server has closed the connection.
alco commented 9 months ago

@andrezimpel Fly supports IPv6, so I"m not sure why it's failing for you. Are you running the latest version of Electric?

andrezimpel commented 9 months ago

@alco my bad! The issue was that prisma could not introspect the database. That's why I encountered an error. Unfortunately I did not get the prisma error which would have given me a hint on the issue. It's working fine now!