search

PeerDB-io / peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
https://peerdb.io
Other
2.17k stars 88 forks source link

Deleting a mirror hangs in UI when target tables are manually deleted first #2015

Closed jonkiq closed 3 weeks ago

jonkiq commented 1 month ago

local peerdb When you delete target tables (in clickhouse) and then try to delete the mirror in peerdb, the UI hangs and the mirror can't be deleted. This is a guess because after I added the table manually in clickhouse, the mirror could be deleted in peerdb.

Amogh-Bharadwaj commented 3 weeks ago

Hi @jonkiq ! Thanks for sharing :) PeerDB does the following as part of drop mirror: On postgres:

  1. Drops the replication slot [if created by peerdb]
  2. Drops the publication [if created by peerdb]

On clickhouse:

  1. Drops [if exists] the _peerdb_raw_table
  2. Deletes relevant rows from some tables in internal catalog

Raw table dropping I doubt would have been the issue since we do IF EXISTS and I'm assuming the clickhouse user had drop permissions I think maybe it was actually waiting for the slot to get dropped ? (this can take a minute or two sometimes)

jonkiq commented 3 weeks ago

I'll see if I can figure out what caused it to hang and make a new ticket if I find anything interesting. Thanks @Amogh-Bharadwaj