search

SeaQL / sea-streamer

🌊 A real-time stream processing toolkit for Rust
https://www.sea-ql.org/SeaStreamer/
Apache License 2.0
263 stars 9 forks source link

Kafka SaslOptions #8

Closed tyt2y3 closed 1 year ago

tyt2y3 commented 1 year ago

Fixes #5

tyt2y3 commented 1 year ago

Does this work for you? @hgzimmerman

hgzimmerman commented 1 year ago

As far as my limited understanding of SASL goes, it looks good.

My example had an additional bootstrap.servers option set, which I'm unsure if is needed for SASL authentication to work. (The bootstrap server(s) provides metadata used to route a client to a particular node(s) to connect to for a topic, among other things). I'm unsure how commonly used that option is or if it is worth making a special function for it.

I think if that proves to be necessary, its best followed up on separately with a set_bootstrap_server(&str) function.

tyt2y3 commented 1 year ago

Thanks. Actually the "bootstrap.servers" key is set for every connection with the values coming from StreamerUri. May be your bootstrap.servers string is not compatible with Url?

https://github.com/SeaQL/sea-streamer/blob/5f552b69aee74f5c4bc89d82161a919b6c1cbb97/sea-streamer-kafka/src/consumer.rs#L646

https://github.com/SeaQL/sea-streamer/blob/5f552b69aee74f5c4bc89d82161a919b6c1cbb97/sea-streamer-kafka/src/producer.rs#L358

hgzimmerman commented 1 year ago

No, my connection profile was just copied nearly verbatim from https://github.com/confluentinc/librdkafka/issues/4119#issuecomment-1361958501 and I wasn't aware that the config option was already set elsewhere in sea-streamer-kafka. I believe that the url I was using works anyways.

This is good news to me, in that I get to remove some redundant code and config values.

tyt2y3 commented 1 year ago

Cool! I am going to make a patch release this week end, including this and your previous PR to redis.

tyt2y3 commented 1 year ago

Released in https://github.com/SeaQL/sea-streamer/releases/tag/0.2.1