toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync
https://pgsync.com
MIT License
1.1k stars 172 forks source link

Neither pgsync itself nor bootstrap command are showing logs but the pgsync schema on console. #507

Open d3vr10 opened 7 months ago

d3vr10 commented 7 months ago

PGSync version: 3.0.0

Postgres version: 16.1

Elasticsearch/OpenSearch version: 8.11.1

Redis version: 7.2.3

Python version: 3.11.6

Problem Description: I run pgsync with DOCKER through the runserver.sh script you provided in the repo plus set the verbosity mode for both (bootstrap and pgsync programs). Either way, I endup with just the schema entries as indentated on console. Like this:

Attaching to pgsync-1 pgsync-1 | wait-for-it.sh: waiting 60 seconds for postgres:5432 pgsync-1 | wait-for-it.sh: postgres:5432 is available after 0 seconds pgsync-1 | wait-for-it.sh: waiting 60 seconds for es01:9200 pgsync-1 | wait-for-it.sh: es01:9200 is available after 0 seconds pgsync-1 | wait-for-it.sh: waiting 60 seconds for redis:6379 pgsync-1 | wait-for-it.sh: redis:6379 is available after 0 seconds pgsync-1 | 2023-11-30 20:59:41.646:WARNING:pgsync.sync: Checkpoint file not found: /pgsync/.rental_rental pgsync-1 | - public.film pgsync-1 | |- public.language pgsync-1 | |- public.category pgsync-1 | |- public.actor pgsync-1 | - public.inventory pgsync-1 | |- public.staff pgsync-1 | | - public.address pgsync-1 | - public.payment pgsync-1 | - public.customer pgsync-1 | - public.address pgsync-1 | - public.city pgsync-1 | - public.country pgsync-1 | - public.film pgsync-1 | |- public.language pgsync-1 | |- public.category pgsync-1 | |- public.actor pgsync-1 | - public.inventory pgsync-1 | |- public.staff pgsync-1 | | - public.address pgsync-1 | - public.payment pgsync-1 | - public.customer pgsync-1 | - public.address pgsync-1 | - public.city pgsync-1 | - public.country

After this, the program ends and there are no further logs on the process itself.

There are two concerns: One, I played one of your examples that use the dvdrental sample database. It took 25 to 26 minutes to bootstrap and sync the db into elasticsearch (There is only one ES node)? Is this expected to last this long?

And second, where could I retrieve the logs from? Does pgsync dump any logs on its operations, (went through the docs and saw no trace of it)?

Message (if any):