yugabyte / debezium-connector-yugabytedb

A Debezium CDC connector for the YugabyteDB database
https://docs.yugabyte.com/stable/explore/change-data-capture/using-logical-replication/yugabytedb-connector/
Apache License 2.0
12 stars 8 forks source link

Not able to setup basic CDC #324

Open saisadashivaK opened 9 months ago

saisadashivaK commented 9 months ago

Hi, I have setup a single node cluster on one my PC using - yugabyted start Next, I used - yb-admin -master_addresses 0.0.0.0:7100 create_change_data_stream ysql.yugabyte The produced stream id was copied by me. Next, I built the debezium yugabyte connector as per instructions from this link

After running curl like specified - curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" \ localhost:8083/connectors/ \ -d '{ "name": "ybconnector", "config": { "connector.class": "io.debezium.connector.yugabytedb.YugabyteDBConnector", "database.hostname":"'", "database.port":"5433", "database.master.addresses": "'':7100", "database.user": "yugabyte", "database.password": "yugabyte", "database.dbname" : "yugabyte", "database.server.name": "dbserver1", "table.include.list":"public.reviews", "database.streamid":"", "snapshot.mode":"never" } }'

However, I am getting {"error_code": 500, "message": "Request timed out"}. Also I am getting Error while trying to get the tablet list to poll for CDC; will attempt to retry 2 of 5 after 60000 milli-seconds. Exception: {} org.yb.client.NonRecoverableException: Too many attempts:.. How to get this working??

vaibhav-yb commented 8 months ago

Hi @saisadashivaK I believe you are running the docker containers as specified in the pipeline. Can you please specify what is the version of Yugabyte you're using?

Also, since you're running the containers in the docker container, you'll need yugabyted running on the IP of your machine which could be something like 192.168.x.x most probably. Another thing you will need is specifying the IP in the hostname and master addresses for the connector configuration in the curl command.

You could also ping us in our community Slack https://communityinviter.com/apps/yugabyte-db/register to take this issue forward.

saisadashivaK commented 8 months ago

I am using Yugabyte 2.19.3.0. I had specified IP in yugabyted as 0.0.0.0. Currently, I am using a local distributed cluster and CDC from cec-examples repository seems to be working fine. I will ping on slack if required. Thank you for responding