Asynchronous commit causes a consumer group session timed out

[mickprev]

Hi!

I have been using the messenger-kafka bundle for some time now and after seeing the time it took to commit a message synchronously, I wanted to switch to asynchronous. So I passed the 'async' parameter to true and got the following errors when I started the message consumption:

SESSTMOUT|rdkafka#consumer-1| [thrd:main]: Consumer group session timed out (in join-state started) after 30004 ms without a successful response from the group coordinator (broker 6, last error was Success): revoking assignment and rejoining group
REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/6: Timed out 0 in-flight, 0 retry-queued, 1 out-queue, 0 partially-sent requests
FAIL|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator:subdomain.confluent.cloud:9092: 1 request(s) timed out: disconnect (after 60822ms in state UP)
ERROR|rdkafka#consumer-1| [thrd:app]: rdkafka#consumer-1: GroupCoordinator:subdomain.confluent.cloud:9092: 1 request(s) timed out: disconnect (after 60822ms in state UP)
REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/6: Timed out OffsetCommitRequest in flight (after 60090ms, timeout #0)
REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/6: Timed out OffsetCommitRequest in flight (after 60090ms, timeout #1)
REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/6: Timed out OffsetCommitRequest in flight (after 60090ms, timeout #2)
REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/6: Timed out OffsetCommitRequest in flight (after 60090ms, timeout #3)
REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/6: Timed out OffsetCommitRequest in flight (after 60090ms, timeout #4)
REQTMOUT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/6: Timed out 3131 in-flight, 0 retry-queued, 27878 out-queue, 0 partially-sent requests
FAIL|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator:subdomain.confluent.cloud:9092: 31009 request(s) timed out: disconnect (after 60169ms in state UP)
ERROR|rdkafka#consumer-1| [thrd:app]: rdkafka#consumer-1: GroupCoordinator:subdomain.confluent.cloud:9092: 31009 request(s) timed out: disconnect (after 60169ms in state UP)

My Configuration

consumer:
    dsn: '%env(KAFKA_DSN)%'
    options:
        commitAsync: true
        kafka_conf:
            enable.auto.offset.store: 'false'
            group.id: '%env(KAFKA_GROUP)%'
            heartbeat.interval.ms: '3000'
            max.poll.interval.ms: '60000'
            sasl.mechanisms: 'PLAIN'
            sasl.password: '%env(KAFKA_PASSWORD)%'
            sasl.username: '%env(KAFKA_USERNAME)%'
            security.protocol: 'SASL_SSL'
            session.timeout.ms: '30000'
            ssl.endpoint.identification.algorithm: 'https'
        receiveTimeout: 10000
        topic:
            name: '%env(KAFKA_TOPIC)%'
        topic_conf:
            auto.offset.reset: 'earliest'
    retry_strategy:
        max_retries: 0
    serializer: App\Serializer

Versions

• librdkafka version: 1.5.3 (same errors in 1.7)
• koco/messenger-kafka: v0.14

@KonstantinCodes any idea? :)