sarama.NewConsumer, what group will kafka use?

[usernameisnull]

Versions

Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly.

Sarama	Kafka	Go
v1.25.0	2.0.0 (Commit:3402a8361b734732)	go version go1.12.14 windows/amd64

Configuration

What configuration values are you using for Sarama and Kafka?

 package main

import (
    "fmt"
    "log"
    "os"
    "os/signal"
    "time"

    "github.com/Shopify/sarama"
    "github.com/alecthomas/log4go"
)

func main() {

    defer func() {
        time.Sleep(time.Second)
        log4go.Warn("[main] consumer quit over!")
        log4go.Global.Close()
    }()

    // sarama的logger
    sarama.Logger = log.New(os.Stdout, fmt.Sprintf("[%s]", "consumer"), log.LstdFlags)

    config := sarama.NewConfig()
    config.Version = sarama.V2_0_0_0
    config.Consumer.Offsets.Initial = sarama.OffsetNewest

    client, err := sarama.NewConsumer([]string{"192.168.198.145:9092"}, config)
    if err != nil {
        panic(err)
    }

    pc, err := client.ConsumePartition("topic.sarama", 0, sarama.OffsetNewest)
    if err != nil {
        panic(err)
    }
    defer func() {
        if err := client.Close(); err != nil {
            log.Fatalln(err)
        }
    }()
    signals := make(chan os.Signal, 1)
    signal.Notify(signals, os.Interrupt)

    consumed := 0

ConsumerLoop:
    for {
        select {
        case msg := <-pc.Messages():
            log.Printf("Consumed message offset %d, msg: %v \n", msg.Offset, string(msg.Value))
            consumed++
        case <-signals:
            break ConsumerLoop
        }
    }
}

Logs

When filing an issue please provide logs from Sarama and Kafka if at all possible. You can set sarama.Logger to a log.Logger to capture Sarama debug output.

logs: CLICK ME

``` ```

Problem Description

I run command in the machine where kafka is:

./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
KMOffsetCache-ubuntu18

1. KMOffsetCache-ubuntu18 is the group of my application use?
2. when I run application on mulitiple machine to connect to the kafka, still only one group KMOffsetCache-ubuntu18, the multiple application use one group?
3. the topic (as in the code: topic.sarama ) only one partition, but all applications can get the message,so:
   • multi application use same group and treated as same client? that impossible, only one consumer can consume the message, and I run these applications on different machines!
   • multi application use their own group, why only KMOffsetCache-ubuntu18 get?

[usernameisnull]

another talk: what the default consumer group id is?

[usernameisnull]

I get sth, in the code I specify the partition, so multiple application can cosume the message

[gonejack]

Sarama 之前是不支持消费组的，现在的版本有，官方有示例：

https://github.com/Shopify/sarama/blob/master/examples/consumergroup/main.go

但之前的使用经验下来感觉并不稳定，有一些怪问题，在某大型企业内部踩过坑。

Sarama has no support for consumer group before, the one merged by pull request is not yet stable now in my experience.

[usernameisnull]

Sarama 之前是不支持消费组的，现在的版本有，官方有示例：

https://github.com/Shopify/sarama/blob/master/examples/consumergroup/main.go

但之前的使用经验下来感觉并不稳定，有一些怪问题，在某大型企业内部踩过坑。

Sarama has no support for consumer group before, the one merged by pull request is not yet stable now in my experience.

那kafka本身是怎么处理这种没有指定消费组的行为的呢?

[gonejack]

单客户端消费不需要组，组只是用来协调多客户端的。

[usernameisnull]

单客户端消费不需要组，组只是用来协调多客户端的。 不,我也是多个应用程序连上去的,这也算多客户端吧

[antsbean]

Sarama 之前是不支持消费组的，现在的版本有，官方有示例：

https://github.com/Shopify/sarama/blob/master/examples/consumergroup/main.go

但之前的使用经验下来感觉并不稳定，有一些怪问题，在某大型企业内部踩过坑。

Sarama has no support for consumer group before, the one merged by pull request is not yet stable now in my experience.

我们目前也在生产环境使用，目前来看，还是稳定，你可以说下具体的问题吗？

[optimistic9527]

@antsbean 当创建多个不同的名称的consumer-group会导致出现无法消费的问题

[gonejack]

@antsbean 有些旧集群会直接连接不上，和 sarama 版本还有关系的，报错比较奇怪，当时没记录下来，直接降低 sarama 版本解决了。

[ghost]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the master branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[hemmmapart]

mark 一下，希望有大佬解答一下

[gonejack]

单消费者没有组的概念，单消费者消费时必须要指定消费哪个数据分片，像提问者贴的代码里就有这一句。

pc, err := client.ConsumePartition("topic.sarama", 0, sarama.OffsetNewest)
    if err != nil {
        panic(err)
    }

如果是消费组的话，这一句是不存在的，服务器自动帮你分配了哪一块数据交给组内的哪个消费者去读取。

所以回答就是没有组，因为只是新建了一个消费者，不是消费组。

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[dnwe]

Closing as believed to have been resolved