Can the interval for requesting data from Kafka be changed?

[dmitry-drozdov]

Hi there, we are using this wonderful library, but are having trouble understanding how the fetcher works. It seems like the documentation lacks details on this.

At the lowest library level, how do you fetch data from Kafka? Is there something that runs periodically? I'm thinking of the following scenario: request data from Kafka -> wait for N seconds -> request data from Kafka -> wait ...

If so, can we control this interval (N seconds mentioned above)?

For better understanding, could you also please answer the following question. Let's say we have the reader

 kafkago.NewReader(kafkago.ReaderConfig{
    Brokers: brokers,
    Topic:   topic,
    GroupID: groupId,
    Dialer:  &kafkago.Dialer{
        Timeout:   10 * time.Second,
        DualStack: true,
    },
})

and call reader.ReadMessage(ctx) in the infinite loop, simply reading and then proceeding to the next iteration. So if someone starts sending messages continuously (in a infinite loop as well), will we receive data without delay? Or, for example, will the library handle messages only at fixed intervals, like every N seconds?

[ansoncht]

First time here, but to my understanding, ReadMessage(ctx) reads a new message whenever it is available with minimal delay. You don't control fetch intervals directly.

Want to know other people's thoughts too