cloudevents / sdk-go

Go SDK for CloudEvents
https://cloudevents.github.io/sdk-go/
Apache License 2.0
806 stars 217 forks source link

[kafka_confluent] should propagate partitionkey on an event level #1086

Open bread-amammay opened 1 month ago

bread-amammay commented 1 month ago

Issue

Currently you have to interact with https://github.com/cloudevents/sdk-go/blob/7fef294789916760157377cb07dc5bd55d31e562/protocol/kafka_confluent/v2/option.go#L137-L151 to set the partition key on an outgoing event. If you have multiple events are you sending on a single context you have to keep swapping values.

Proposal

I propose that the paradigm from kafka_sarama is copied over where it checks to see if the partition key is set on an a cloud event extension, if it is, set that on the outgoing record. https://github.com/cloudevents/sdk-go/blob/f1c0d0aac5ac90027b6e5504a44a14fa8f1985d9/protocol/kafka_sarama/v2/write_producer_message.go#L38-L48

yanmxa commented 1 month ago

Thanks @bread-amammay! I see that both implementations use the same way to inject the partitionKey for the Sender. The difference is that the sarma implementation also adds partitionKey by event extension. But what if the key exists in both context and extension?

I agree with the idea of supporting the partitionKey at the event level. But we need to document their priorities.

Do you want to have a try? :)