crossplane-contrib / provider-kafka

Crossplane provider for Kafka
Apache License 2.0
27 stars 30 forks source link

CannotCreateExternalResource EOF error #53

Closed lariskovski closed 1 year ago

lariskovski commented 1 year ago

What happened?

After following the documentation I expected to be able to deploy a topic resource. Instead I keep getting the erros below on the kafka-provider's logs:

[ERROR] unable to request api versions; broker: seed 0, err: EOF
[provider-kafka] 2023-01-30T14:40:29.762ZDEBUG   controller-runtime.manager.events       Warning {"object": {"kind":"Topic","name":"sample-topic","uid":"xxxx","apiVersion":"topic.kafka.crossplane.io/v1alpha1","resourceVersion":"1200071675"}, "reason": "CannotCreateExternalResource", "message": "EOF"}

topic.yaml

apiVersion: topic.kafka.crossplane.io/v1alpha1
kind: Topic
metadata:
  name: sample-topic
spec:
  forProvider:
    replicationFactor: 1
    partitions: 1
  providerConfigRef:
    name: kafka-stage-provider

How can we reproduce it?

I followed the README steps, deployed the topic resource and looked for the provider logs

What environment did it happen in?

Crossplane version: helm.sh/chart: crossplane-1.7.0 Kubernetes distribution: GKE 1.21 Kafka: Confluent Cluster

jograca commented 1 year ago

We saw similar behavior when Crossplane was not able to communicate with our Kafka clusters. Are you able to reproduce this with the Kafka setup we recommend here? Or anything indicating a communication problem on your Kafka cluster side?

klinux commented 1 year ago

The error in topic creation is:

Message: observe failed: cannot get topic spec from topic client: cannot list topics: EOF

There's any config to talk with Kafka confluent SASL?

Thank you

jograca commented 1 year ago

SASL/PLAIN is the only mechanism we wrote and currently support in the project.

lariskovski commented 1 year ago

@jograca thanks for your attention.

We were able to make it work with Confluent using with the insecureSkipVerify:

{
  "brokers":[
    "xxxxx"
   ],
   "sasl":{
     "mechanism":"PLAIN",
     "username":"yyyy",
     "password":"zzzz"
   },
   "tls": {
    "insecureSkipVerify": true
   }
}