replicase / pgcapture

A scalable Netflix DBLog implementation for PostgreSQL
Apache License 2.0
220 stars 31 forks source link

pgcapture extension also parse temp table #71

Open KennyChenFight opened 23 hours ago

KennyChenFight commented 23 hours ago

Hi @rjuju The issue now is that the pgcapture extension is capturing queries related to temp tables that PostgreSQL itself creates (for example, during REFRESH MATERIALIZED VIEW). See below example:

postgres=# select id, query, tags from pgcapture.ddl_logs order by id desc limit 3;
28 | DROP TABLE pg_temp_827.pg_temp_20855_2, pg_temp_827.pg_temp_20855                    | {"DROP TABLE"}
27 | ALTER TABLE pg_temp_827.pg_temp_20855_2 ADD COLUMN newdata pg_temp_827.pg_temp_20855 | {"ALTER TABLE"}
26 | CREATE TEMP TABLE pg_temp_827.pg_temp_20855_2 (tid pg_catalog.tid)                   | {"CREATE TABLE"}

This causes the downstream PostgreSQL to receive the same event, leading to errors such as 'temp table does not exist.' What would be a good way to filter out this type of query? Do you suggest any good solution? thx!

rjuju commented 23 hours ago

I will work on a fix for that soon (I'm traveling around for the national day holiday).

it should be straightforward to handle most of the commands, but DROP is a bit more tricky as you can drop both regular and temp tables in the same command. it can be handled too though (I actually already fixed that problem in some internal extension at work).

KennyChenFight commented 23 hours ago

I will work on a fix for that soon (I'm traveling around for the national day holiday).

it should be straightforward to handle most of the commands, but DROP is a bit more tricky as you can drop both regular and temp tables in the same command. it can be handled too though (I actually already fixed that problem in some internal extension at work).

I am looking forward to! thx again!