search

kedacore / keda

KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes
https://keda.sh
Apache License 2.0
8.59k stars 1.09k forks source link

Support Kafka SASL MSK IAM authentication #3431

Closed pragmaticivan closed 1 year ago

pragmaticivan commented 2 years ago

Proposal

MSK (Kafka Managed Services from AWS) supports IAM for authentication via SASL.

ref: https://github.com/segmentio/kafka-go/issues/661

Use-Case

ServiceAccount should be able to be used for Kafka authentication.

Anything else?

No response

tomkerkhove commented 2 years ago

Fair ask! Are you willing to contribute this?

pragmaticivan commented 2 years ago

I might be able to help but will take quite some time (Still refreshing my skills with go) 😂 If anyone else has the need for this feature asap.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 1 year ago

This issue has been automatically closed due to inactivity.

pragmaticivan commented 1 year ago

Should this be closed? cc @zroubalik

Manicben commented 10 months ago

Understand that the experimental scaler is based on kafka-go and supports MSK IAM via the custom SASL mechanism, but just as an FYI, AWS released an alternative based on SASL-OAUTHBEARER which works with all Go-based Kafka libraries.

Just wanted to highlight this in case it made any difference to the future of the non-experimental and experimental Kafka scalers.

AWS announcement from Nov 2023 GH repo including examples for Sarama

JorTurFer commented 10 months ago

@dttung2905 @zroubalik . Wasn't the support to MSK IAM the only reason for the new scaler? I mean, could we unify both into the original one adding support to MSK IAM thanks to this?

dttung2905 commented 9 months ago

@JorTurFer sorry for the late reply. Its been a busy week for me. Iirc, supporting MSK IAM was a main reason, besides better support and documentation as outlined in https://github.com/kedacore/keda/issues/4692.

What's our policy when it comes to deprecating scaler ? Do we follow N-2 cycle too? I took at look at the doc but its not clear too me on scaler deprecation https://github.com/kedacore/governance/blob/main/DEPRECATIONS.md

JorTurFer commented 9 months ago

The first step is to figure out if the experimental scaler brings extra features (not only now but also in the future) or its just for AWS

dttung2905 commented 9 months ago

The first step is to figure out if the experimental scaler brings extra features (not only now but also in the future) or its just for AWS

Its a really good point to bring up during our next community meet up! We can discuss more indept there :muscle: