search

toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync
https://pgsync.com
MIT License
1.11k stars 174 forks source link

Checkpoint file not found error #551

Open deepakbhatt-busy opened 3 weeks ago

deepakbhatt-busy commented 3 weeks ago

PGSync version: 3.1.0

Postgres version: Postgres 15

Elasticsearch/OpenSearch version: 8.14.1

Redis version: 7.2.1

Python version: 3.9.6

Problem Description: I am trying to prepare a POC having one table in my database and want to sync with index in my local. Below is my schema json.

[ { "database": "test_db", "index": "leads", "nodes": { "table": "leads", "columns": [ "id", "lead_title", "assignee1", "assignee2", "assignee3", "created_by" ] } } ]

Error Message (if any): When i am running bootstrap --config schema.json command. I am getting below logs

2024-07-05 12:55:43.008:INFO:pgsync.utils: Settings: 2024-07-05 12:55:43.008:INFO:pgsync.utils: Schema : schema.json 2024-07-05 12:55:43.008:INFO:pgsync.utils: ----------------------------------------------------------------- 2024-07-05 12:55:43.008:INFO:pgsync.utils: Checkpoint: 2024-07-05 12:55:43.008:INFO:pgsync.utils: Path: /Users/deepak/Projects/PG-Sync 2024-07-05 12:55:43.008:INFO:pgsync.utils: Postgres: 2024-07-05 12:55:43.008:INFO:pgsync.utils: URL: postgresql+psycopg2://postgres:@localhost/postgres 2024-07-05 12:55:43.008:INFO:pgsync.utils: Elasticsearch: 2024-07-05 12:55:43.008:INFO:pgsync.utils: URL: http://elastic:****@localhost 2024-07-05 12:55:43.008:INFO:pgsync.utils: Redis: 2024-07-05 12:55:43.008:INFO:pgsync.utils: URL: redis://localhost:6379/0 2024-07-05 12:55:43.008:INFO:pgsync.utils: ----------------------------------------------------------------- 2024-07-05 12:55:43.059:INFO:elastic_transport.transport: GET http://localhost:9200/ [status:200 duration:0.012s]

image

Pavangj959 commented 2 weeks ago

Hi @deepakbhatt-busy, It is just a warning, you can ignore it.

deepakbhatt-busy commented 2 weeks ago

Hi @Pavangj959 . Thanks for your reply. I tried to run the 2nd command after running the 1st command (i.e., pgsync --config schema.json), but now I'm getting the following error 2024-07-08 10:22:12.112:ERROR:pgsync.base: (psycopg2.errors.ConfigurationLimitExceeded) all replication slots are in use HINT: Free one or increase max_replication_slots. Please find attached screenshot for reference.

image

I believe this is happening because when we executed the 1st command, it created a replication slot, and as mentioned in the documentation, my database configuration has max_replication_slots = 1.

and if i am deleting existing replication slot and running 2nd command again it is giving error.

RuntimeError: Replication slot "test_db_leads" does not exist. Make sure you have run the "bootstrap" command.

image