Segfault consuming large number of Kafka topics when using `musl`

[fpytloun]

Vector Version

vector 0.15.0 (x86_64-unknown-linux-musl 23b6c9e 2021-07-13)

Vector Configuration File

    [sources.in_kafka_access]
      type = "kafka"
      bootstrap_servers = "kafka:9092"
      group_id = "vector"
      topics = ["^fluentd.(oneapp|anotherapp)_.*$"]
      librdkafka_options."client.id" = "${HOSTNAME}"
      librdkafka_options."client.rack" = "myregion"
      librdkafka_options."group.instance.id" = "${HOSTNAME}"
      topic_key = "_topic"
      partition_key = "_partition"
      offset_key = "_offset"

      tls.enabled = true
      tls.ca_file = "/etc/ssl/server_ca.crt"
      tls.crt_file = "/etc/ssl/client.crt"
      tls.key_file = "/etc/ssl/client.key"

Debug Output

Expected Behavior

Vector should subscribe to any number of topics and just work. This might be also issue in rdkafka.

Actual Behavior

Vector segfaults when consumer group is settled.

Example Data

Additional Context

Running with hundreds of topics works fine but as number of topics grown to 1200, it started to segfault after restart. Until that, it was working properly.

Reducing number of consumed topics fixes the issue.

References

[fpytloun]

I just found out that this issue doesn't happen with glibc (using Debian image)

[jszwedko]

Thanks for testing @fpytloun . That additional information is useful.