failed to list connectors from Kafka connect cluster

[arthurfedorov]

Hi everyone, first of all, i just want to say thanks for the cool service.

Deployed by helm chart Redpanda appVersion: v24.2.2 Redpanda console appVersion: v2.4.6

I have almost default values on it, except ingress values for console, and custom docker image for connectors. Dockerfile:

FROM redpandadata/connectors:v1.0.26

LABEL version=mongodb-kafka-connect:v1.0.5

COPY mongo-kafka-connect-1.13.0-all.jar /opt/kafka/redpanda-plugins/mongodb-kafka-connect/mongo-kafka-connect-1.13.0-all.jar

After helm install redpanda -n redpanda, everything is up

k get pods -n redpanda
NAME                                   READY   STATUS    RESTARTS       AGE
redpanda-0                             2/2     Running   0              118m
redpanda-1                             2/2     Running   0              118m
redpanda-2                             2/2     Running   0              118m
redpanda-connectors-65b5595784-wtqkt   1/1     Running   1 (118m ago)   118m
redpanda-console-8598665998-x9zww      1/1     Running   0              115m

When i open Connectors tab in Redpanda console, i see available connectors.

After I create any of the Mongodb connectors, it is created, but it is not in the interface. The logs in pod show the following error

│ {"level":"warn","ts":"2024-10-14T17:05:19.753Z","msg":"failed to list connectors from Kafka connect cluster","cluster_name":"connectors","cluster_address":"http://redpanda-connectors.re │
│ dpanda.svc.cluster.local:8083","error":"Get \"http://redpanda-connectors.redpanda.svc.cluster.local:8083/connectors?expand=info&expand=status\": context deadline exceeded"}              │
│ {"level":"warn","ts":"2024-10-14T17:05:19.917Z","msg":"failed to list connectors from Kafka connect cluster","cluster_name":"connectors","cluster_address":"http://redpanda-connectors.re │
│ dpanda.svc.cluster.local:8083","error":"Get \"http://redpanda-connectors.redpanda.svc.cluster.local:8083/connectors?expand=info&expand=status\": context deadline exceeded"}              │
│ {"level":"warn","ts":"2024-10-14T17:06:14.084Z","msg":"failed to list connectors from Kafka connect cluster","cluster_name":"connectors","cluster_address":"http://redpanda-connectors.re │
│ dpanda.svc.cluster.local:8083","error":"Get \"http://redpanda-connectors.redpanda.svc.cluster.local:8083/connectors?expand=info&expand=status\": context deadline exceeded"}              │
│ {"level":"warn","ts":"2024-10-14T17:13:17.970Z","msg":"failed to list connectors from Kafka connect cluster","cluster_name":"connectors","cluster_address":"http://redpanda-connectors.re │
│ dpanda.svc.cluster.local:8083","error":"Get \"http://redpanda-connectors.redpanda.svc.cluster.local:8083/connectors?expand=info&expand=status\": context deadline exceeded"}              │
│ {"level":"warn","ts":"2024-10-14T17:13:18.138Z","msg":"failed to list connectors from Kafka connect cluster","cluster_name":"connectors","cluster_address":"http://redpanda-connectors.re │
│ dpanda.svc.cluster.local:8083","error":"Get \"http://redpanda-connectors.redpanda.svc.cluster.local:8083/connectors?expand=info&expand=status\": context deadline exceeded"}   

But when I go into Redpanda console pod and do curl http://redpanda-connectors.redpanda.svc.cluster.local:8083/connectors?expand=info&expand=status, the list of connectors is visible.

/app $ curl http://redpanda-connectors.redpanda.svc.cluster.local:8083/connectors?expand=info&expand=status
/app $ {"mongodb-source-connector-xz6j":{"info":{"name":"mongodb-source-connector-xz6j","config":{"connector.class":"com.mongodb.kafka.connect.MongoSourceConnector","publish.full.document.only":"false","output.schema.infer.value":"false","startup.mode":"latest","topic.creation.default.partitions":"1","connection.password":"my_password","output.format.value":"json","publish.full.document.only.tombstone.on.delete":"false","connection.username":"my_user","output.format.key":"json","topic.creation.enable":"true","key.converter.schemas.enable":"true","database":"my_database","connection.uri":"mongodb://my_host:27017/my_database?replicaSet=rs&ssl=false","name":"mongodb-source-connector-xz6j","topic.creation.default.replication.factor":"-1","value.converter.schemas.enable":"true","connection.url":"mongodb://my_host:27017/my_database?replicaSet=rs&ssl=false","post.processor.chain":"com.mongodb.kafka.connect.sink.processor.DocumentIdAdder","value.converter":"org.apache.kafka.connect.storage.StringConverter","key.converter":"org.apache.kafka.connect.storage.StringConverter"},"tasks":[{"connector":"mongodb-source-connector-xz6j","task":0}],"type":"source"}}}

Can you please tell me where I made a mistake?

[weeco]

Hey @arthurfedorov , could you show us the Kafka connect config in Console? I think I've seen a similar case where a timeout of 0 was used which meant the request immediately times out as soon as it was sent.