search

utilitywarehouse / go-pubsub

A simple abstraction for message publishing and consumption in Go
MIT License
20 stars 10 forks source link

Multiple topics subscription #57

Open Gobonoid opened 6 years ago

Gobonoid commented 6 years ago

Currently go-pubsub doesn't allow passing list of topics for single kafka consumer.

type MessageSourceConfig struct {
    ConsumerGroup            string
    Topic                    string
    Brokers                  []string
    Offset                   int64
    MetadataRefreshFrequency time.Duration
    Version                  *sarama.KafkaVersion
}

It would be nice if this would be 

type MessageSourceConfig struct {
    ConsumerGroup  string
    Topic                    []string
    Brokers                 []string
    Offset                   int64
    MetadataRefreshFrequency time.Duration
    Version                  *sarama.KafkaVersion
}
aneshas commented 5 years ago

Is this a relatively straightforward change as it looks since sarama cluster already takes a slice of strings for topics cluster.NewConsumer(mq.brokers, mq.consumergroup, []string{mq.topic}, config) ?

mjgarton commented 5 years ago

The questions that come to mind are: 1) Can this be done in a way that doesn't compromise the goals of go-pubsub (making all brokers "look" the same) 2) If so, is the gain worth the effort and potential added complexity?