Is your feature request related to a problem? Please describe.
Changefeed users may have network connectivity problems from CRDB to Kafka. This is difficult to troubleshoot. The error returned is often kafka: client has run out of available brokers to talk to (Is your cluster reachable?), which I understand to result from any network problem.
Further, Cloud users do not have network troubleshooting tools (ping, nslookup, traceroute) because they can’t ssh into Cloud nodes. And, they don’t have access to logs, where they might cat *.log | grep sarama.
We might go further and add things like WITH dns, traceroute, ping etc, which would not be Kafka-specific. And then iterate on other sinks such as webhook and pubsub.
blathers-crl[bot]
commented
1 year ago
Is your feature request related to a problem? Please describe.
Changefeed users may have network connectivity problems from CRDB to Kafka. This is difficult to troubleshoot. The error returned is often
kafka: client has run out of available brokers to talk to (Is your cluster reachable?), which I understand to result from any network problem.Further, Cloud users do not have network troubleshooting tools (
ping,nslookup,traceroute) because they can’tsshinto Cloud nodes. And, they don’t have access to logs, where they mightcat *.log | grep sarama.Problems might also include certs and auth.
Describe the solution you'd like
The ability to test Kafka connectivity from the CRDB process, via the SQL client. The Disaster Recovery team has implemented similar for cloud storage: https://github.com/cockroachdb/cockroach/pull/97517
The solution may be an extension of the above to accept
kafka://URLs. We’d need some way to generate Kafka-specific workload.See also kcat for inspiration: https://docs.confluent.io/platform/current/clients/kafkacat-usage.html
We might go further and add things like
WITH dns, traceroute, pingetc, which would not be Kafka-specific. And then iterate on other sinks such as webhook and pubsub.Possbile duplicate of https://github.com/cockroachdb/cockroach/issues/60178
Jira issue: CRDB-25245 Epic CRDB-6667