jonathanyuechun
commented
3 years ago I updated the issue, the sensor problem was because of insufficient default RBAC permission over argo-worflow-sa service account.
Open jonathanyuechun opened 3 years ago
jonathanyuechun
commented
3 years ago I updated the issue, the sensor problem was because of insufficient default RBAC permission over argo-worflow-sa service account.
whynowy
commented
3 years ago The failure should be because the partition ID is not available. However, it's an issue that the error is not printed out.
cbuckley01
commented
3 years ago The failure should be because the partition ID is not available. However, it's an issue that the error is not printed out.
Right that is a real/larger issue, trying to debug connection and other sarama/kafka without the lower level errors is not fun.
github-actions[bot]
commented
3 years ago This issue has been automatically marked as stale because it has not had any activity in the last 60 days. It will be closed if no further activity occurs. Thank you for your contributions.
MATRIX4284
commented
3 years ago This issue is happening as the partition number is not created in the kafka topic that is being used as a part of event source .
GET THE CORRECT PARTITION ID: Just enter the shell of ur kafka pod using
1.kubectl exec -it kafka-sever-pod-id /bin/bash
2.Go to the $KAFKA_HOME/bin folder.
3.Run the command below to see which PARTITION_ID is available
./kafka_topics.sh --describe --zookeeper k8s-svc-host:port -topic topic1
Topic: topic1 Partition: 0 Leader: 1001 Replicas: 1001 Isr: 1001 Topic: topic1 Partition: 1 Leader: 1001 Replicas: 1001 Isr: 1001
See whether the partition that is mnetioned in the kafka-event-source.yaml exists for the topic being used in kafka_event_src.
You will see that the partition that is mentioned in event source kafka.yaml is not present and that is the reason the kafka event source ceases to run .
if i does not exist then use the partition that is available for the topic i.e. Partition: 0 & 1 is availabe for topic1.
Edit the kafka_event_source manifest file https://github.com/argoproj/argo-events/tree/master/examples/event-sources/kafka.yaml as below:
MATRIX4284
commented
3 years ago apiVersion: argoproj.io/v1alpha1 kind: EventSource metadata: name: kafka spec: kafka: example:
url: kafka.argo-events:9092
# name of the kafka topic
topic: topic1
# jsonBody specifies that all event body payload coming from this
# source will be JSON
jsonBody: true
# partition id
**### AS BOTH 0 & 1 partition id is available for topic1 so we can set either 0 / 1 as partition ID **
partition: "0"
# optional backoff time for connection retries.
# if not provided, default connection backoff time will be used.
connectionBackoff:
# duration in nanoseconds, or strings like "3s", "2m". following value is 10 seconds
duration: 1000000
# how many backoffs
steps: 5
# factor to increase on each step.
# setting factor > 1 makes backoff exponential.
factor: 2
jitter: 0.2
Describe the bug
Hi, i'm testing out the kafka event source
and kafka sensorexamples: https://argoproj.github.io/argo-events/eventsources/setup/kafka.Event Source
To Reproduce Steps to reproduce the behavior:
Used minikube as POC environment.
Kafka installation
https://strimzi.io/docs/operators/in-development/quickstart.html#assembly-evaluation-str
Tried on argo-events and eventbus:
Expected behavior Events are published to event source and kafka sensor is dispatched
Screenshots
missing kafka sensor podEnvironment (please complete the following information):
Additional context
i can see the topic being created by inspecting my broker with
kafka-topics.shMessage from the maintainers:
If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.