search

vmware-archive / kubeless

Kubernetes Native Serverless Framework
https://kubeless.io
Apache License 2.0
6.86k stars 754 forks source link

Kubeless topic command don't work with custom Kafka #1031

Open LudovitVarga opened 5 years ago

LudovitVarga commented 5 years ago

Is this a BUG REPORT or FEATURE REQUEST?: It's a bug.

What happened: Scenario: I have existing Kafka+Zookeeper cluster (installed by this Helm chart) and i configured Kubeless Kafka Trigger to work this cluster. Trigger works as expected when i publish to topic by kafka-topics.sh inside Kafka pod. But when i use command from my PC kubeless topic list --kafka-namespace kafka, i got following error:

websocket.Dial wss://192.168.99.110:8443/api/v1/namespaces/kafka/pods/kafka-0/exec?command=bash&command=%2Fopt%2Fbitnami%2Fkafka%2Fbin%2Fkafka-topics.sh&command=--zookeeper&command=zookeeper.kafka%3A2181&command=--list&container=broker&stderr=true&stdout=true: bad status

What you expected to happen: I expect to see list of Kafka topics.

Anything else we need to know?: Same result for kubeless topic create/delete/publish.

Environment:

andresmgot commented 5 years ago

Hi @LudovitVarga,

The kubeless topic command was mainly created for testing purposes and that's why it assumes certain things about the Kafka environment. You can workaround it though. See: https://github.com/kubeless/kubeless/issues/775#issuecomment-391634961

LudovitVarga commented 5 years ago

The kubeless topic command was mainly created for testing purposes and that's why it assumes certain things about the Kafka environment.

It is mentioned somewhere in documentation? This information can save time for other developers. Anyway, it is sad that so strong assumption is made here because this command can be very helpful for testing, what is as you said, main purpose of this command.

andresmgot commented 5 years ago

I agree and it's probably not explained in the documentation. We can modify the command, AFAIK the only assumption that we do is that the pod is running with the label kubeless: kafka but we would need to verify that. Would you be able to spend some time verifying that?

LudovitVarga commented 5 years ago

Would you be able to spend some time verifying that?

Yes. I'll do it as soon as possible.

LudovitVarga commented 5 years ago

Now i noticed that command makes another assumption: zookeeper has name zookeeper and is in same namespace as kafka.

LudovitVarga commented 5 years ago

I agree and it's probably not explained in the documentation. We can modify the command, AFAIK the only assumption that we do is that the pod is running with the label kubeless: kafka but we would need to verify that. Would you be able to spend some time verifying that?

I can confirm that label kubeless: kafka is required for kubeless topic command.

andresmgot commented 5 years ago

Thanks @LudovitVarga, maybe you are able to provide a patch for the command? That assumption should be easy to change in commands changing it for flags. The code that uses it is for example this:

https://github.com/kubeless/kubeless/blob/master/cmd/kubeless/topic/topicCreate.go#L74

LudovitVarga commented 5 years ago

Thanks @LudovitVarga, maybe you are able to provide a patch for the command? That assumption should be easy to change in commands changing it for flags. The code that uses it is for example this:

https://github.com/kubeless/kubeless/blob/master/cmd/kubeless/topic/topicCreate.go#L74

Only problem/challenge here is my zero experience with Go lang :) If i find some time i can take a look at code but i can't promise it.