Segmentation fault when running functional phpunit tests which spawn many consumers consecutively

[kandrejevs]

Description

Hi, I have weird issue when running functional tests in phpunit with process isolation. We have legacy codebase where we try to integrate kafka and many tests are end to end integration tests, so having messages into them looks like sending message, calling command to consume them, continue doing something else and assert results.

after few hundred tests I start getting %3|1641384126.072|ERROR|rdkafka#consumer-55| [thrd:kafka:9092/bootstrap]: 2/2 brokers are down

and soon some random tests fail with Segmentation fault (core dumped) as far I was able to debug, seems that issue comes from broker.

Any help or tips how to avoid this are appreciated.

How to reproduce

Create phpunit test where message is produced and consumed and data provider with 1000 or more elements. Run this test After some time it should fail randomly with segmentation fault

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

• [x] librdkafka version (release number or git tag): 1.8.2 | 1.6.2
• [x] Apache Kafka version: 3.0.0
• [x] librdkafka client configuration:

  'auto.offset.reset', 'earliest'
  'log.connection.close', 'false'
  'queue.buffering.max.ms', '1'

• [x] Operating system: ubuntu 18.04
• [ ] Provide logs (with debug=.. as necessary) from librdkafka
• [ ] Provide broker log excerpts
• [ ] Critical issue

[milindl]

Hey, is this still an issue with the latest version 2.5.0? Could you attach a backtrace of the segmentation fault? (By opening the dumped core with gdb and using bt to grab the backtrace).