search

dimitri / pgcopydb

Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids)
Other
1.19k stars 78 forks source link

Issue of jobs in online #238

Open citusvamsi opened 1 year ago

citusvamsi commented 1 year ago

Hi Team,

Migrate database from Azure flexy server PostgreSQL to Azure Cosmos DB for PostgreSQL .

I am using pgcopydb for online migration to migrate database from Azure flexy server PostgreSQL to Azure Cosmos DB for PostgreSQL. Source side(Azure Flexy server postgresql) ,I have created a job that is inserting data into a dummy table for every 2 mins.

After running pgocpydb clone --follow getting below issue

image

Command use for online migration:

pgcopydb clone --follow --table-jobs  8 --skip-extensions \
  --source "host={server name} port=5432 dbname=postgres user=postgres password={your password}" \
  --target "host={server name} port=5432 dbname=citus    user=citus    password=}your password}" \
  --restart --no-owner --drop-if-exists --no-acl \
  --no-comments --filter filter.ini

Even I excluded the table job_run_details using filter.ini file also getting same issue

filter.ini

[exclude-schema]
pghero
monitoring
new_relic
repack
partman
cron
[exclude-table]
job_run_details
cron.job_run_details

Any solution?

dimitri commented 1 year ago

Hi @citusvamsi ; please share the logs output from running pgcopydb as text. As an image it makes it harder to read (font sizes, font itself, colors and contrast, etc) and I can't even copy paste. I still can see an error about permission denied for table job_run_details which apparently you have listed in the exclude-table section.

At the moment pgcopydb support for filtering has not been added to the “online” parts of it: logical decoding is not filtered yet.