segmentio / kafka-go

Kafka library in Go
MIT License
7.49k stars 775 forks source link

The offset is submitted, but each execution still starts from the beginning and repeats #1318

Open ChenghuaMi opened 1 month ago

ChenghuaMi commented 1 month ago

func main() { r := kafka.NewReader(kafka.ReaderConfig{ Brokers: []string{"localhost:9092"}, Topic: "my-topic", Partition: 0, MaxBytes: 10e6, // 10MB }) for { m, err := r.ReadMessage(context.Background()) if err != nil { break } fmt.Printf("message at offset %d: %s = %s\n", m.Offset, string(m.Key), string(m.Value)) r.CommitMessages(context.Background(), m) // commit offset }

if err := r.Close(); err != nil {
    log.Fatal("failed to close reader:", err)
}

}

I submitted the offset, but each time I execute the program, it starts from the beginning, resulting in repeated consumption.

===============First Execution=========== message at offset 0: = one! message at offset 1: = two! message at offset 2: = three!

============== Second Execution ========= message at offset 0: = one! message at offset 1: = two! message at offset 2: = three!

luffyao commented 2 weeks ago

i think you must use the consumer group to do this.