[CustomPartitioner] Help required

[sakib-malik-7417]

I was following up this blog on how to use custom partitioner in consumer group clients and add custom userdata in JoinGroupMetadata request and fetch this in each subsequent metadata fetch requests

I was wondering if we can achieve the same for cooperative-sticky partitioner (because it actually uses the UserData field present in the ConsumerMemberMetadata struct and desirialization can override any appends to StickyMemberMetadata used in UserData)

[twmb]

Technically, I don't believe you can. You'd need to "fork" the partitioner / wrap the partitioner. The problem is that to be forward compatible, the UserData can contain extra junk at the end of it. This is also how the protocol supports rolling upgrades, or members of the same balancer type all working in the group successfully even if they have different versions: new versions use new data at the end of the UserData field, old members ignore that data.

If you added new data on an existing version, then a new version may try to parse that data and it'll result in an error.

I think the safest way to do this would be to write your own balancer that wraps the sticky balancer. In the balance function, first parse your extra data (at the beginning?) and then call the wrapped balancer with the original underlying UserData.

[sakib-malik-7417]

Thanks for the help @twmb closing this as completed