2ndQuadrant / pglogical

Logical Replication extension for PostgreSQL 17, 16, 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.
http://2ndquadrant.com/en/resources/pglogical/
Other
1.01k stars 153 forks source link

Missing Records Updated inside a Function #201

Open badinvestor opened 5 years ago

badinvestor commented 5 years ago

I have a somewhat large data warehouse database (1.5TB) that I want to upgrade from 9.5.5 to 10.6. I have data that was updated via function call on the pub. The data was never replicated to the sub node. There are no errors in the logs and replication of all other data was successful.

Has anyone ran across this before? Could it be related to this issue?

These are the steps used to create the pub and sub.

 session_replication_role
--------------------------
 origin
--Create replication node on source db:
SELECT pglogical.create_node(node_name := 'dw_provider', dsn := 'host=127.0.0.1 port=5432 dbname=dw'); 

-- Add tables to the publication:
SELECT pglogical.replication_set_add_all_tables('default', '{dwmart}');
SELECT pglogical.replication_set_add_all_tables('default', '{dw}');

-- Add sequences:
SELECT pglogical.replication_set_add_all_sequences(set_name := 'default', schema_names := '{dwmart}', synchronize_data := true );
SELECT pglogical.replication_set_add_all_sequences(set_name := 'default', schema_names := '{dw}', synchronize_data := true );

-- Create subscription on sub node:
SELECT pglogical.create_node(node_name := 'dw_subscriber', dsn := 'host=127.0.0.2 port=5433 dbname=dw');
SELECT pglogical.create_subscription(subscription_name := 'dw_subscription', provider_dsn := 'host=127.0.0.1 port=5432 dbname=dw', synchronize_structure := true);
ringerc commented 5 years ago

What was the function?

Written in what language?

Is there any chance it set session_replication_role=replica ? Or configured the replication origin? If so, that would be the expected outcome.

I can't help you without some reasonable level of detail - at bare minimum all the involved versions and the relevant function + its invocation, preferably a full repro test case.

badinvestor commented 5 years ago

Pglogical sets session_replication_role=replica with each copy command was my understanding. How would that prevent updates called with a function from being replicated? The function is PL/pgSQL.

The function is quite long and involved but the tl;dr of it is that it selects from a set of staging tables does a small amount of transformation of data and either inserts or updates records in subsequent tables based on long string of business logic. The inserts all were replicated but the none of the updates (about 14,000 records in all)

AgnosticDBA commented 4 years ago

did you get a resolution for this issue? we are having a similar issue using pglogical to AWS (they use pglogical 2.2.0)