vmware-archive / kafka-trigger

Kubernetes CRD controller for Kafka topic as event source for Kubeless functions
Apache License 2.0
28 stars 34 forks source link

updated sarama versions and base kubeless project version to fix the … #13

Closed dimagoldin closed 5 years ago

dimagoldin commented 5 years ago

…CRC error when using kafka version 2.0

Issue Ref: [Issue number related to this PR or None] https://github.com/kubeless/kubeless/issues/970 Description: On consuming messages from kafka version 2.0, the kafka client fails with CRC mismatch errors.

[PR Description] Version changes to sarama library to latest version that includes fixes the the bug in question. Version change to the kubeless base project to latest stable (v1.0.0) TODOs:

andresmgot commented 5 years ago

Hi @dimagoldin,

I fixed a couple of issues related to the latest kubeless but now the CI is failing because the compilation of the sarama library:

#!/bin/bash -eo pipefail
make VERSION=${CONTROLLER_TAG} binary
CGO_ENABLED=1 ./script/binary
Removing Old Kafka trigger controller binary
Build Kafka trigger controller binary
# github.com/kubeless/kafka-trigger/vendor/github.com/Shopify/sarama
vendor/github.com/Shopify/sarama/config.go:200:14: undefined: BalanceStrategy
Makefile:29: recipe for target 'binary' failed
make: *** [binary] Error 2
Exited with code 2

Have you tried to compile this locally? Maybe it's necessary to update something else?

dimagoldin commented 5 years ago

@andresmgot

These are the steps i took; updated glide.yaml with the new versions glide update -v make kafka-controller-image docker tag kafka-controller-image myregistry/kafka-controller-image docker push myregistry/kafka-controller-image

deleted the old deployment from my k8s cluster changed k8s definitions with the the new image and the corresponding secret deployed the new deployment to k8s cluster created kafka trigger published messages to topic, checked logs and saw everything is consumed and triggered perfectly

Just to make sure, i deployed the old image again, published messages and saw CRC errors. Redeployed new one, published same messages, everything triggered correctly.

glide update or glide install (without -v) didnt work, no idea why, first time im working with glide (or golang)

andresmgot commented 5 years ago

Hi @dimagoldin, the problem with glide update is that is trying to update all the dependencies, which is causing errors. Probably is working for you because you have some local dependencies that are being used. In order to fix it run:

rm -rf glide.lock vendor/
glide install -v
git add .
git commit -m "Update deps"

That will update the sarama library leaving the rest untouched. Can you try that?

dimagoldin commented 5 years ago

@andresmgot you were right, the build passes now. :)

andresmgot commented 5 years ago

Thanks @dimagoldin !