Application with a producer that uses default partitioner crashes in debug mode

[vpa1977]

Description

Using nuget 1.9.3 with dotnet 6.0.110 on Ubuntu 22.10 Also reproduced on Debian 11 with dotnet 6.0.402

Debugging an application that uses a producer with a default partitioner is impossible since it crashes in the partitioner callback

Stack trace of thread 2991594:
                #0  0x00007f50ffa7d911 n/a (n/a + 0x0)
                #1  0x00007f5164a4671b rd_kafka_msg_partitioner (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x4671b)
                #2  0x00007f5164a4b348 rd_kafka_topic_assign_uas (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x4b348)
                #3  0x00007f5164a4ccc2 rd_kafka_topic_metadata_update (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x4ccc2)
                #4  0x00007f5164a4d9b0 rd_kafka_topic_metadata_update2 (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x4d9b0)
                #5  0x00007f5164ac3287 rd_kafka_parse_Metadata (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0xc3287)
                #6  0x00007f5164a6e523 rd_kafka_handle_Metadata (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x6e523)
                #7  0x00007f5164a5d1c4 rd_kafka_buf_callback (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x5d1c4)
                #8  0x00007f5164a6a783 rd_kafka_op_handle_std (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x6a783)
                #9  0x00007f5164a6a7d8 rd_kafka_op_handle (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x6a7d8)
                #10 0x00007f5164a627a8 rd_kafka_q_serve (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x627a8)
                #11 0x00007f5164a29f9c rd_kafka_thread_main (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0x29f9c)
                #12 0x00007f5164abc2f7 _thrd_wrapper_function (/home/vladimirp@hamilton.first-watch.co.nz/git/the-base/KafkaCrashReportSimple/bin/Debug/net6.0/runtimes/linux-x64/native/librdkafka.so + 0xbc2f7)
                #13 0x00007f5178efd402 start_thread (libc.so.6 + 0x90402)
                #14 0x00007f5178f8c590 __clone3 (libc.so.6 + 0x11f590)

How to reproduce

Run the program below (an argument is a bootstrap server):

using Confluent.Kafka;

Console.WriteLine("Hello, World!");
{
    using var producer = new ProducerBuilder<string, string>(new ProducerConfig()
    {
        BootstrapServers = args[0]
    }).SetDefaultPartitioner((topic, count, data, isKey) =>
    {
        return new Partition(0);
    }).Build();

    producer.Produce("topic", new Message<string, string>()
    {
        Key = "test", Value = "test"
    }, report =>
    {
        Console.WriteLine("Delivered");
    } );
    producer.Flush();
}

Console.WriteLine("Success");

In run mode it prints:

Hello, World!
Delivered
Success

In debug mode it crashes with the stack trace above.

Checklist

Please provide the following information:

• [x] A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
• [x] Confluent.Kafka nuget version. 1.9.3
• [x] Apache Kafka version. 2.7.0
• [x] Client configuration.
• [x] Operating system. (Ubuntu 22.10, 22.04, Debian 11)
• [x] Provide logs (with "debug" : "..." as necessary in configuration). - no logs
• [x] Provide broker log excerpts. - no logs
• [ ] Critical issue. - not critical, can be worked around in dev by disabling the partitioner (?)

[vpa1977]

Appears to be some kind of runtime issue - the parameters passed in native code to the partitioner callback are identical, but the sample crashes when the debugger is attached.

[grahambunce]

Getting this too on an older version 1.8.2. Tears down the whole process thereby avoiding .net exception handling. Ubuntu 22.04.1 LTS

Setting a partitioner at topic level debugs fine, though obviously this means you need to know all your possible topics upfront.