search

2ndQuadrant / pglogical

Logical Replication extension for PostgreSQL 17, 16, 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.
http://2ndquadrant.com/en/resources/pglogical/
Other
1.01k stars 154 forks source link

Does table replication with row_filter require more memory? #477

Open kostyay opened 5 months ago

kostyay commented 5 months ago

Hi I'm trying to replicate a table with a row_filter defined (start_time > current_date - interval '4 months') and it causes memory usage to spike to 100% which ultimately causes the replication to go down as the process crashed with OOM. Its a table with about 5 mil rows on a 6 vcpu 16 gig Postgres 14 machine. If I try to sync the entire table without row_filter it finishes successfully. I tried creating an index on start_time but it didn't help. Is this the expected behaviour? Is there anything I can do to resolve this (besides adding more memory)? Thanks!

danrough commented 3 months ago

We also experience this issue, in a similar set of circumstances - large table, which we're trying to replicate using a row filter, on an r7g.4xl postgres RDS, and the process brings the whole instance down - the main process gets OOM killed. Without a row filter however, the replication is fine.

Did you find any ways to optimise this, @kostyay?

kostyay commented 1 month ago

Unfortunately not

danrough commented 1 month ago

We moved to using the native replication with a row filter, rather than using the pglogical module, and we have been able to progress with this. So we've now abandoned the use of the pglogical module for anything other than replicating sequences.