Use the RebalanceCb to listen to rebalance events

[james-johnston-thumbtack]

Use the rebalance callback parameter on the SubscribeTopics function instead of listening to callback events from the Poll function itself. This will unblock upgrades to newer versions of the kafka client.

The documentation for "go.application.rebalance.enable" states the following:

Forward rebalancing responsibility to application via the Events()
channel.

Notably, it is not documented to return events from the Poll() function. Despite this, confluent-kafka-go v1.5.2 would do so anyway, and the kafka scheduler was built under this assumption.

However, confluent-kafka-go v1.6.1 refactored the rebalance event handling to support incremental rebalances. As part of this refactoring, confluent-kafka-go no longer returns rebalance events from the Poll() function. Therefore, in order to facilitate any future upgrades of confluent-kafka-go, we must rewrite the code to listen to events using SubscribeTopics instead.

The confluent-kafka-go commit that broke kafka-message-scheduler:

https://github.com/confluentinc/confluent-kafka-go/commit/d5b3a82ad0073d77800961e8bb776e0934abc8b3

[james-johnston-thumbtack]

I would recommend merging https://github.com/etf1/kafka-message-scheduler/pull/42 before this commit. I had some tests failing otherwise.

[fkarakas]

LGTM thank very much @james-johnston-thumbtack you rocks