Unhandled Exception: System.InvalidOperationException: No such configuration property: "produce.offset.report"

[cruxhoo]

Description

An Exception is throw when executing application in docker and the app stops.

Unhandled Exception: System.InvalidOperationException: No such configuration property: "produce.offset.report" at Confluent.Kafka.Impl.SafeConfigHandle.Set(String name, String value) at Confluent.Kafka.Producer.<>c__DisplayClass33_0.<.ctor>b__6(KeyValuePair2 kvp) at System.Collections.Generic.List1.ForEach(Action1 action) at Confluent.Kafka.Producer..ctor(IEnumerable1 config) at Confluent.Kafka.Producer2..ctor(IEnumerable1 config, SerializerGenerator1 keySerializerGenerator, SerializerGenerator1 valueSerializerGenerator) at tw_public_connector.Services.KafkaProducer.ConfigureProducer() in /app/Services/KafkaProducer.cs:line 42

How to reproduce

Create an Asp.net core app and install confluent kafka. Dockerfile:

FROM microsoft/dotnet:2.2-aspnetcore-runtime AS base
FROM microsoft/dotnet:2.2-sdk
WORKDIR /app
COPY *.csproj ./
RUN dotnet restore

RUN apt-get update \
    && apt-get install -y --no-install-recommends libgdiplus libc6-dev librdkafka-dev \
    && rm -rf /var/lib/apt/lists/*
COPY . ./
RUN dotnet publish -c Release -o out
ENTRYPOINT ["dotnet", "out/tw-public-connector.dll"]

initializing the producer(Here is where the exception occurs):

private void ConfigureProducer()
{
    var config = new ProducerConfig()
    {
        BootstrapServers = _config["Kafka:Host"],
        GroupId = _config["Kafka:GroupId"]
     };
    _producer = new Producer<Null, string>(config);
 }

Checklist

Please provide the following information:

• [x] Confluent.Kafka nuget version: 1.0.0-beta2
• [x] Apache Kafka version: docker image spotify/kafka latest
• [x] Client configuration:
• [x] Operating system: Docker image microsoft/dotnet:2.2
• [ ] Provide logs (with "debug" : "..." as necessary in configuration)
• [ ] Provide broker log excerpts
• [x] Critical issue

[cruxhoo]

I was able to solve this by installing an older version of librdkafka (0.11.4). This is the code required in dockerfile

ENV LIBRDKAFKA_VERSION 0.11.4
RUN curl -Lk -o /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz https://github.com/edenhill/librdkafka/archive/v${LIBRDKAFKA_VERSION}.tar.gz && \
  tar -xzf /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz -C /root && \
  cd /root/librdkafka-${LIBRDKAFKA_VERSION} && \
  ./configure --prefix /usr && make && make install && make clean && ./configure --clean

[mhowlett]

interesting. first, you don't need to / shouldn't install the librdkafka-dev package - Confluent.Kafka references the librdkafka.redist nuget package which provides an appropriate librdkafka build for the version you reference. i'm guessing the error is occurring because the librdkafka-dev librdkafka binary is being referenced not the librdkafka.redist one, but I'm confused as to why that would be happening and also I wouldn't expect that error... finally, 'produce.offset.report' is no longer set in the 1.0.x branch, but that change was made after 1.0-beta2 was cut. Look out for a new version soon.

[mhowlett]

ahh, if the librdkafka binary is not compatible with the base image used by the dotnet:2.2 image, that would explain why it's finding the librdkafka-dev version. i haven't verified this yet.

[mhowlett]

update: I've verified the librdkafka build in librdkafka.redist referenced by confluent.kafka beta2 is compatible with the dotnet 2.2 docker image.