GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.13k stars 943 forks source link

PostGres support for CDC #95

Closed darrenhaken closed 4 years ago

darrenhaken commented 4 years ago

I've stumbled across the article using DataFlow with Debeizum for CDC on MySQL. Would it be possible to also support Postgres on CloudSQL?

I've evaluated Postgres with Debezium directly but annoyingly CloudSQL Postgres doesn't work.

sabhyankar commented 4 years ago

@pabloem

pabloem commented 4 years ago

Thanks for the request @darrenhaken - I'll look into having postgress support for this soon. Let me see if I can hack it together in the next few weeks.

darrenhaken commented 4 years ago

That's awesome! If you need any input on anything do let me know, I look forward to it πŸ˜€

nathaniel-md commented 4 years ago

I would also think this is interesting! just a plus one on this issue! πŸ˜‰

pabloem commented 4 years ago

thanks! - I am starting to work on this. The holdup right now is writing an integration test. I'll keep this issue updated.

nathaniel-md commented 4 years ago

This whole thread is basically asking for the same thing as well, though they seem to be wanting to manage debezium themselves so are asking for extra rights on the postgres cloudsql.

https://issuetracker.google.com/issues/70756171

nathaniel-md commented 4 years ago

Hey! No stress about this, but I wondered if there was a projected timeline for this issue? It would be nice to be able to plan around whether something like this will be coming soon. Thanks!

pabloem commented 4 years ago

I finally got around to pushing out the postgres support! sorry about the delay, I know it took me a while longer than promised.

pabloem commented 4 years ago

Note the instructions here: https://github.com/GoogleCloudPlatform/DataflowTemplates/tree/master/v2/cdc-parent#using-the-debezium-connector-with-postgres

pabloem commented 4 years ago

FYI @griscz @nathaniel-md @darrenhaken

darrenhaken commented 4 years ago

Is that a valid URL? I get sent to the project folder

pabloem commented 4 years ago

yes, the Using the Debezium connector with Postgres subsection of the README

pabloem commented 4 years ago

(note that multiplexing from a single pubsub topic is supported now as well, so that should ease part of the setup)

nathaniel-md commented 4 years ago

Yay!

Thank you so much! I can’t wait to get started using it and seeing how it works!

On Tue, 9 Jun 2020 at 21:51, Pablo notifications@github.com wrote:

(note that multiplexing from a single pubsub topic is supported now as well, so that should ease part of the setup)

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-641534224, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKFCVA5HRTE53LZCRNF5T7LRV2HENANCNFSM4MGPCMIQ .

--

Nathaniel Joselson

Data Scientist

Docly UK

+46761817020

Docly Healthcare International β€’ www.docly.com

darrenhaken commented 4 years ago

Does this mean Debezium works with Postgres on GCP now or only via DataFlow?

On Tue, 9 Jun 2020, 21:08 nathaniel-md, notifications@github.com wrote:

Yay!

Thank you so much! I can’t wait to get started using it and seeing how it works!

On Tue, 9 Jun 2020 at 21:51, Pablo notifications@github.com wrote:

(note that multiplexing from a single pubsub topic is supported now as well, so that should ease part of the setup)

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-641534224 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AKFCVA5HRTE53LZCRNF5T7LRV2HENANCNFSM4MGPCMIQ

.

--

Nathaniel Joselson

Data Scientist

Docly UK

+46761817020

Docly Healthcare International β€’ www.docly.com

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-641542057, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHIY6Z5PB3URB3JJMHVF5TRV2JDLANCNFSM4MGPCMIQ .

gvanlandeghem commented 4 years ago

any news on postgreSQL supporting logical replication? AWS and Azure support this feature.

pabloem commented 4 years ago

sorry, but I have no information on that. The place to track that will be here: https://issuetracker.google.com/70756171

nathaniel-md commented 4 years ago

Hey @pabloem

I am just getting around to testing this now but am coming up on a couple errors I don't understand.

I am trying to deploy on a GCP VM where I am also running a .cloud_sql_proxy connection to a Postgres database.

My .properties file looks like this:

databaseName=gcp-project:europe-west1:test-db
databaseUsername=user
databaseAddress=127.0.0.1
databasePort=3001
gcpProject=gcp-projec
gcpPubsubTopicPrefix=dev_cdc
databaseManagementSystem=postgres
debezium.database.dbname=testdb
whitelistedTables=test-db.testdb.cases,test-db.testdb.users
singleTopicMode=true
databasePassword=password
inMemoryOffsetStorage=true

I created a pub/sub topic called cdc_dev_cdc

Then when trying to run mvn -Ppostgres exec:java -pl cdc-embedded-connector -Dexec.args="config.properties" I get the following error:

org.apache.kafka.connect.errors.ConnectException: Cannot create replication connection
        at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.<init>(PostgresReplicationConnection.java:101)
        at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.<init>(PostgresReplicationConnection.java:47)
        at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$ReplicationConnectionBuilder.build(PostgresReplicationConnection.java:478)
        at io.debezium.connector.postgresql.PostgresTaskContext.createReplicationConnection(PostgresTaskContext.java:111)
        at io.debezium.connector.postgresql.RecordsStreamProducer.<init>(RecordsStreamProducer.java:85)
        at io.debezium.connector.postgresql.RecordsSnapshotProducer.<init>(RecordsSnapshotProducer.java:76)
        at io.debezium.connector.postgresql.PostgresConnectorTask.createRecordProducer(PostgresConnectorTask.java:134)
        at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:113)
        at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:49)
        at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:791)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.debezium.jdbc.JdbcConnectionException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off
        at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initReplicationSlot(PostgresReplicationConnection.java:175)
        at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.<init>(PostgresReplicationConnection.java:93)
        ... 14 more
Caused by: org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off

Do you have any idea what I am doing wrong? Thanks!

gvanlandeghem commented 4 years ago

Are you running the connector against cloud sql postgreSQL?

If so, replication using the native pgoutput mechanism for replication is not yet supported, see open issue https://issuetracker.google.com/issues/130128163

You need LOGIN and REPLICATION roles as database user in the debezium connector to be able to use the WAL based replication mechanism. Unfortunately none of the users created in Cloud Sql postgreSQL have superuser rights nor can they assign LOGIN or REPLICATION roles to created users. You are not allowed either to install other output plugins like WAL2JSON.

The above issue is not scheduled to be solved in 2020.

Alternatives:

On Thu, Jul 9, 2020 at 2:55 AM nathaniel-md notifications@github.com wrote:

Hey @pabloem https://github.com/pabloem

I am just getting around to testing this now but am coming up on a couple errors I don't understand.

I am trying to deploy on a GCP VM where I am also running a .cloud_sql_proxy connection to a Postgres database.

My .properties file looks like this:

databaseName=gcp-project:europe-west1:test-db databaseUsername=user databaseAddress=127.0.0.1 databasePort=3001 gcpProject=gcp-projec gcpPubsubTopicPrefix=dev_cdc databaseManagementSystem=postgres debezium.database.dbname=testdb whitelistedTables=test-db.testdb.cases,test-db.testdb.users singleTopicMode=true databasePassword=password inMemoryOffsetStorage=true

I created a pub/sub topic called cdc_dev_cdc

Then when trying to run mvn -Ppostgres exec:java -pl cdc-embedded-connector -Dexec.args="config.properties" I get the following error:

org.apache.kafka.connect.errors.ConnectException: Cannot create replication connection at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:101) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:47) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$ReplicationConnectionBuilder.build(PostgresReplicationConnection.java:478) at io.debezium.connector.postgresql.PostgresTaskContext.createReplicationConnection(PostgresTaskContext.java:111) at io.debezium.connector.postgresql.RecordsStreamProducer.(RecordsStreamProducer.java:85) at io.debezium.connector.postgresql.RecordsSnapshotProducer.(RecordsSnapshotProducer.java:76) at io.debezium.connector.postgresql.PostgresConnectorTask.createRecordProducer(PostgresConnectorTask.java:134) at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:113) at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:49) at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:791) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: io.debezium.jdbc.JdbcConnectionException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initReplicationSlot(PostgresReplicationConnection.java:175) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:93) ... 14 more Caused by: org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off

Do you have any idea what I am doing wrong? Thanks!

β€” You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-655832479, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABKHRYMBUXBRTKKWUO6ZBTR2UIPDANCNFSM4MGPCMIQ .

--

kind regards,

Geert Van Landeghem

Big Data Consultant

DataCrunchers.eu

Steenweg van Grembergen 27, 9200 Grembergen

btw: BE 0872.859.349

gsm: +32 477 759533

web1: http://www.datacrunchers.eu

web2: http://www.foundation.be

http://twitter.com/gvanlandeghem

http://be.linkedin.com/in/geertvanlandeghem

Skype: geertvanlandeghem

nathaniel-md commented 4 years ago

Hey! Thanks for the quick response!

Yes, I am using cloud sql postgres. Talking through your alternatives:

  • Switch to cloud sql mysql I don't think this going to happen soon...
  • DIY hosting of postgreSQL (kubernetes) Do you mean I can host a replica that the debezium can connect to or are you talking about moving the database from cloudsql to an unmanaged version?
  • Switch to the jdbc connector What would that mean?
  • use timestamp + primary key column How would I do this?
  • use soft deletes (because connector doesnt detect deletes) What does this mean?

Thanks!

Nate

On Thu, Jul 9, 2020 at 8:04 AM gvanlandeghem notifications@github.com wrote:

Are you running the connector against cloud sql postgreSQL?

If so, replication using the native pgoutput mechanism for replication is not yet supported, see open issue https://issuetracker.google.com/issues/130128163

You need LOGIN and REPLICATION roles as database user in the debezium connector to be able to use the WAL based replication mechanism. Unfortunately none of the users created in Cloud Sql postgreSQL have superuser rights nor can they assign LOGIN or REPLICATION roles to created users. You are not allowed either to install other output plugins like WAL2JSON.

The above issue is not scheduled to be solved in 2020.

Alternatives:

  • Switch to cloud sql mysql
  • DIY hosting of postgreSQL (kubernetes)
  • Switch to the jdbc connector
  • use timestamp + primary key column
  • use soft deletes (because connector doesnt detect deletes)

On Thu, Jul 9, 2020 at 2:55 AM nathaniel-md notifications@github.com wrote:

Hey @pabloem https://github.com/pabloem

I am just getting around to testing this now but am coming up on a couple errors I don't understand.

I am trying to deploy on a GCP VM where I am also running a .cloud_sql_proxy connection to a Postgres database.

My .properties file looks like this:

databaseName=gcp-project:europe-west1:test-db databaseUsername=user databaseAddress=127.0.0.1 databasePort=3001 gcpProject=gcp-projec gcpPubsubTopicPrefix=dev_cdc databaseManagementSystem=postgres debezium.database.dbname=testdb whitelistedTables=test-db.testdb.cases,test-db.testdb.users singleTopicMode=true databasePassword=password inMemoryOffsetStorage=true

I created a pub/sub topic called cdc_dev_cdc

Then when trying to run mvn -Ppostgres exec:java -pl cdc-embedded-connector -Dexec.args="config.properties" I get the following error:

org.apache.kafka.connect.errors.ConnectException: Cannot create replication connection at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:101) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:47) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$ReplicationConnectionBuilder.build(PostgresReplicationConnection.java:478) at io.debezium.connector.postgresql.PostgresTaskContext.createReplicationConnection(PostgresTaskContext.java:111) at io.debezium.connector.postgresql.RecordsStreamProducer.(RecordsStreamProducer.java:85) at io.debezium.connector.postgresql.RecordsSnapshotProducer.(RecordsSnapshotProducer.java:76) at io.debezium.connector.postgresql.PostgresConnectorTask.createRecordProducer(PostgresConnectorTask.java:134) at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:113) at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:49) at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:791) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: io.debezium.jdbc.JdbcConnectionException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initReplicationSlot(PostgresReplicationConnection.java:175) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:93) ... 14 more Caused by: org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off

Do you have any idea what I am doing wrong? Thanks!

β€” You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-655832479 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AABKHRYMBUXBRTKKWUO6ZBTR2UIPDANCNFSM4MGPCMIQ

.

--

kind regards,

Geert Van Landeghem

Big Data Consultant

DataCrunchers.eu

Steenweg van Grembergen 27, 9200 Grembergen

btw: BE 0872.859.349

gsm: +32 477 759533

web1: http://www.datacrunchers.eu

web2: http://www.foundation.be

http://twitter.com/gvanlandeghem

http://be.linkedin.com/in/geertvanlandeghem

Skype: geertvanlandeghem

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-655921007, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKFCVAYFTSV6MP42KXNOPVDR2VMYTANCNFSM4MGPCMIQ .

--

Nathaniel Joselson

Data Scientist

Docly UK

+46761817020

Docly Healthcare International β€’ www.docly.com

pabloem commented 4 years ago

I am sorry, but as pointed out by @gvanlandeghem, Cloud SQL Postgres does not work well with Debezium because it does not support logical decoding - and installing WAL2JSON. : /

gvanlandeghem commented 4 years ago

On Thu, Jul 9, 2020 at 11:47 AM nathaniel-md notifications@github.com wrote:

Hey! Thanks for the quick response!

Yes, I am using cloud sql postgres. Talking through your alternatives:

  • Switch to cloud sql mysql I don't think this going to happen soon...

ok :)

  • DIY hosting of postgreSQL (kubernetes) Do you mean I can host a replica that the debezium can connect to or are you talking about moving the database from cloudsql to an unmanaged version?

moving the database from cloudsql to a DIY postgreSQl (eg running as docker container in kubernetes)

  • Switch to the jdbc connector What would that mean?

Switching to the https://docs.confluent.io/current/connect/kafka-connect-jdbc/index.html connector, this connector doesn't use the binlog (mysql) or wal (postgresql) but polls the database to detect new or updated records (polling) Of course listening to a database log mechanism is much more efficient than continually polling the database at intervals.

  • use timestamp + primary key column How would I do this?

Polling database changes can best be done using a combination of timestamp (filled in when insert or update is executed) and primary key field.

  • use soft deletes (because connector doesnt detect deletes) What does this mean?

Since this connector cannot detect deletes (it polls for changes on the table data not on the log) you can implement a "soft delete" mechanism where a deleted column (eg _deleted) in the record is flagged to true. Your application will no longer delete records but update the deleted flag in the database row and your application will not use records flagged as deleted. Hence, your application logic needs to be modified. Downstream the soft delete events can be used as real delete events.

At one of our customers we use connectors to synchronize mongoDB, sql server, postgreSQL and mysql databases with Google BigQuery datasets. It allows our customer to build a view on top of all company data for the first time. We transform and anonymize data along the way using our KStream+SMT libraries. In BigQuery we create views on top of the event data and build data marts on top of these, reporting is implemented using Google Data Studio. AI models are being built on top of the integrated datasets. These AI models will be later integrated into business processes. All this speeds up the process to deliver data to data scientists and to incorporate insights they deliver. Later on real-time data marts (360Β° views) could be created by moving the current ETL batch code in bigquery to the streaming layer. These datamarts could contain denormalized views to be used by an API layer (no need for joins since data is denormalized) for the external world and/or in a platform architecture. Since all ETL code runs in kubernetes (realtime kstream+smt and batch kubeflow) we could migrate to another cloud vendor if wanted. However, replacing BigQuery as datalake won't be easy.

Get the data flowing between all (data and application) components in your EA using Kafka :)

Thanks!

Nate

On Thu, Jul 9, 2020 at 8:04 AM gvanlandeghem notifications@github.com wrote:

Are you running the connector against cloud sql postgreSQL?

If so, replication using the native pgoutput mechanism for replication is not yet supported, see open issue https://issuetracker.google.com/issues/130128163

You need LOGIN and REPLICATION roles as database user in the debezium connector to be able to use the WAL based replication mechanism. Unfortunately none of the users created in Cloud Sql postgreSQL have superuser rights nor can they assign LOGIN or REPLICATION roles to created users. You are not allowed either to install other output plugins like WAL2JSON.

The above issue is not scheduled to be solved in 2020.

Alternatives:

  • Switch to cloud sql mysql
  • DIY hosting of postgreSQL (kubernetes)
  • Switch to the jdbc connector
  • use timestamp + primary key column
  • use soft deletes (because connector doesnt detect deletes)

On Thu, Jul 9, 2020 at 2:55 AM nathaniel-md notifications@github.com wrote:

Hey @pabloem https://github.com/pabloem

I am just getting around to testing this now but am coming up on a couple errors I don't understand.

I am trying to deploy on a GCP VM where I am also running a .cloud_sql_proxy connection to a Postgres database.

My .properties file looks like this:

databaseName=gcp-project:europe-west1:test-db databaseUsername=user databaseAddress=127.0.0.1 databasePort=3001 gcpProject=gcp-projec gcpPubsubTopicPrefix=dev_cdc databaseManagementSystem=postgres debezium.database.dbname=testdb whitelistedTables=test-db.testdb.cases,test-db.testdb.users singleTopicMode=true databasePassword=password inMemoryOffsetStorage=true

I created a pub/sub topic called cdc_dev_cdc

Then when trying to run mvn -Ppostgres exec:java -pl cdc-embedded-connector -Dexec.args="config.properties" I get the following error:

org.apache.kafka.connect.errors.ConnectException: Cannot create replication connection at

io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:101)

at

io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:47)

at

io.debezium.connector.postgresql.connection.PostgresReplicationConnection$ReplicationConnectionBuilder.build(PostgresReplicationConnection.java:478)

at

io.debezium.connector.postgresql.PostgresTaskContext.createReplicationConnection(PostgresTaskContext.java:111)

at

io.debezium.connector.postgresql.RecordsStreamProducer.(RecordsStreamProducer.java:85)

at

io.debezium.connector.postgresql.RecordsSnapshotProducer.(RecordsSnapshotProducer.java:76)

at

io.debezium.connector.postgresql.PostgresConnectorTask.createRecordProducer(PostgresConnectorTask.java:134)

at

io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:113)

at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:49) at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:791) at

java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at

java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at

java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:834) Caused by: io.debezium.jdbc.JdbcConnectionException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off at

io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initReplicationSlot(PostgresReplicationConnection.java:175)

at

io.debezium.connector.postgresql.connection.PostgresReplicationConnection.(PostgresReplicationConnection.java:93)

... 14 more Caused by: org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "evard_dev", SSL off

Do you have any idea what I am doing wrong? Thanks!

β€” You are receiving this because you commented. Reply to this email directly, view it on GitHub <

https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-655832479

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AABKHRYMBUXBRTKKWUO6ZBTR2UIPDANCNFSM4MGPCMIQ

.

--

kind regards,

Geert Van Landeghem

Big Data Consultant

DataCrunchers.eu

Steenweg van Grembergen 27, 9200 Grembergen

btw: BE 0872.859.349

gsm: +32 477 759533

web1: http://www.datacrunchers.eu

web2: http://www.foundation.be

http://twitter.com/gvanlandeghem

http://be.linkedin.com/in/geertvanlandeghem

Skype: geertvanlandeghem

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-655921007 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AKFCVAYFTSV6MP42KXNOPVDR2VMYTANCNFSM4MGPCMIQ

.

--

Nathaniel Joselson

Data Scientist

Docly UK

+46761817020

Docly Healthcare International β€’ www.docly.com

β€” You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/95#issuecomment-656028036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABKHR2T4AVJJZVA3NBR4ZTR2WGZZANCNFSM4MGPCMIQ .

--

kind regards,

Geert Van Landeghem

Big Data Consultant

DataCrunchers.eu

Steenweg van Grembergen 27, 9200 Grembergen

btw: BE 0872.859.349

gsm: +32 477 759533

web1: http://www.datacrunchers.eu

web2: http://www.foundation.be

http://twitter.com/gvanlandeghem

http://be.linkedin.com/in/geertvanlandeghem

Skype: geertvanlandeghem