search

eulerto / wal2json

JSON output plugin for changeset extraction
BSD 3-Clause "New" or "Revised" License
1.33k stars 161 forks source link

Guidance using wal2json on AWS? #75

Closed nasajoey closed 6 years ago

nasajoey commented 6 years ago

AWS RDS newbie here.

Trying to follow this tutorial on streaming data from Postgres RDS:

It relies on wal2json.

The RDS docs say that wal2json is supported in the version of Postgres I am running (9.6.9):

When running show rds.extensions; I get no wal2json:

address_standardizer,address_standardizer_data_us,bloom,btree_gin,btree_gist,chkpass,citext,cube,dblink,dict_int,dict_xsyn,earthdistance,fuzzystrmatch,hll,hstore,hstore_plperl,intagg,intarray,ip4r,isn,log_fdw,ltree,orafce,pgaudit,pgcrypto,pgrouting,pgrowlocks,pgstattuple,pg_buffercache,pg_freespacemap,pg_hint_plan,pg_prewarm,pg_repack,pg_stat_statements,pg_trgm,pg_visibility,plcoffee,plls,plperl,plpgsql,pltcl,plv8,postgis,postgis_tiger_geocoder,postgis_topology,postgres_fdw,prefix,sslinfo,tablefunc,test_parser,tsearch2,tsm_system_rows,tsm_system_time,unaccent,uuid-ossp

I can't find any instructions on getting wal2json enabled on RDS, but everywhere says it should be possible.

Any guidance on getting it working on RDS? I feel like I must be missing something obvious.

Thanks in advance.

nasajoey commented 6 years ago

Well, I just kept going with the tutorial and stuff worked. Still a little confused as to where wal2json lives and how it got wherever it lives. I'm OK with confusion if still getting results.

lievertz commented 4 years ago

Comment for history: I found this a bit later after looking around at a similar topic. Despite the eulerto github page (colloquially?) calling wal2json an extension, it indeed does not show up in pg_extension. It does not appear to conform to the postgresql extension framework added in 9.1 (https://www.postgresql.org/message-id/1324999772.3291.21.camel@localhost.localdomain).

You will only find it referenced by the column plugin on the view pg_replication_slots (or the function pg_get_replication_slots()). And if you are bothering to do so, you might also want to look at the active boolean to see if the replication slot you are attempting to use this module on is active or not.