Invalid librdkafka version b06ff, expected at least 10502ff

[AyeshaSadiq7]

Description

Hi,

I am trying to dockerise a C# client application in Linux that produces data on a remote Kafka Server by fetching Simulated PMU data from an openPDC application running on a Linux VM.

I am able to run this application using MSBuild command with Mono through the terminal but when I run the same application through the docker file, it gives librdkafka DLL not found or invalid librdkafka version b06ff expected at least 10502ff error depending on the docker file contents.

Note: The dockerized program runs successfully if I remove the 'Produce' function call in the Progam. cs file.

// Error with installing librdkafka-dev through docker file:

Parsing exception: System.Exception: MultiProtocolFrameParser "ReceivedDataFrame" consumer event handler exception: Invalid librdkafka version b06ff, expected at least 10502ff** ---> System.IO.FileLoadException: Invalid librdkafka version b06ff, expected at least 10502ff at Confluent.Kafka.Impl.Librdkafka.Initialize (System.String userSpecifiedPath) [0x00093] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Producer2[TKey,TValue]..ctor (Confluent.Kafka.ProducerBuilder2[TKey,TValue] builder) [0x000cd] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.ProducerBuilder2[TKey,TValue].Build () [0x00000] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Examples.ProducerExample.Program.Produce (System.String topic, Confluent.Kafka.ClientConfig config, System.Object dataObject) [0x00006] in <a74f528a35b74bf08608a527b1c99a05>:0 at Confluent.Kafka.Examples.ProducerExample.Program.parser_ReceivedDataFrame (System.Object sender, GSF.EventArgs1[T] e) [0x000af] in :0 at GSF.PhasorProtocols.MultiProtocolFrameParser.m_frameParser_ReceivedDataFrame (System.Object sender, GSF.EventArgs`1[T] e) [0x0006c] in :0 --- End of inner exception stack trace ---

In this case, I am not able to understand the Invalid librdkafka version b06ff, expected at least 10502ff error, can any one interpret the version number for me, please?**

//Error without installing librdkafka-dev through docker file: Parsing exception: System.Exception: MultiProtocolFrameParser "ReceivedDataFrame" consumer event handler exception: Failed to load the librdkafka native library. ---> System.DllNotFoundException: Failed to load the librdkafka native library. at Confluent.Kafka.Impl.Librdkafka.TrySetDelegates (System.Collections.Generic.List1[T] nativeMethodCandidateTypes) [0x00034] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Impl.Librdkafka.LoadLinuxDelegates (System.String userSpecifiedPath) [0x000ac] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Impl.Librdkafka.Initialize (System.String userSpecifiedPath) [0x0003a] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Producer2[TKey,TValue]..ctor (Confluent.Kafka.ProducerBuilder2[TKey,TValue] builder) [0x000cd] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.ProducerBuilder2[TKey,TValue].Build () [0x00000] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Examples.ProducerExample.Program.Produce (System.String topic, Confluent.Kafka.ClientConfig config, System.Object dataObject) [0x00006] in <73f7781881954796baf4c4278f5b98e7>:0 at Confluent.Kafka.Examples.ProducerExample.Program.parser_ReceivedDataFrame (System.Object sender, GSF.EventArgs1[T] e) [0x000af] in <73f7781881954796baf4c4278f5b98e7>:0 at GSF.PhasorProtocols.MultiProtocolFrameParser.m_frameParser_ReceivedDataFrame (System.Object sender, GSF.EventArgs1[T] e) [0x0006c] in :0

***2nd Way I used to remove the error**

1. I tried installing librdkafka1.6.1 by running the following commands through the terminal.
2. then using 'Confluent.Kafka.Library.Load("/usr/local/lib/librdkafka.so")' in the Progam.cs file but it gives the following error

sudo tar -zxvf librdkafka-1.6.1.tar.gz sud cd librdkafka-1.6.1 sudo ./configure sudo make sudo make install

Unhandled Exception: System.DllNotFoundException: libdl assembly: type: member:(null) at (wrapper managed-to-native) Confluent.Kafka.Impl.Librdkafka+PosixNative.dlopen(string,int) at Confluent.Kafka.Impl.Librdkafka.LoadLinuxDelegates (System.String userSpecifiedPath) [0x00003] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Impl.Librdkafka.Initialize (System.String userSpecifiedPath) [0x0003a] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Library.Load (System.String path) [0x00000] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Examples.ProducerExample.Program.Main (System.String[] args) [0x000b6] in :0 [ERROR] FATAL UNHANDLED EXCEPTION: System.DllNotFoundException: libdl assembly: type: member:(null) at (wrapper managed-to-native) Confluent.Kafka.Impl.Librdkafka+PosixNative.dlopen(string,int) at Confluent.Kafka.Impl.Librdkafka.LoadLinuxDelegates (System.String userSpecifiedPath) [0x00003] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Impl.Librdkafka.Initialize (System.String userSpecifiedPath) [0x0003a] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Library.Load (System.String path) [0x00000] in <8da1dd4a1b55410cbe56aebd9052eef4>:0 at Confluent.Kafka.Examples.ProducerExample.Program.Main (System.String[] args) [0x000b6] in :0

Can you please suggest some solution or otherwise help interpret the Invalid version error? I am new to dotnet and Kafka, might have missed something. Can anyone please help?

How to reproduce

I tried many different ways of writing the docker file, the .csproj file and structuring project hierarchy in many different ways.

Finally, I am now using the minimal Producer example program from https://github.com/confluentinc/confluent-kafka-dotnet/tree/master/examples/Producer and added my required libraries but the error is the same.

I am using UBUNTU 18.04 with Mono 6.12.0.122

// The project file I am using for the Producer example project with the name 'Producer.csproj' is attached

//This is my simplest Docker file (also attached)

From mono COPY . /src WORKDIR /src RUN apt-get update -y RUN nuget install Confluent.Kafka RUN apt-get install -y -V librdkafka-dev RUN msbuild /p:Configuration=Mono /src/Producer.csproj WORKDIR /src/bin EXPOSE 8889 CMD ["mono", "/src/bin/Producer.exe"]

Checklist

Please provide the following information:

• [x] A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file. Note: I am changing files extension to .txt for the attachment purpose only

Program.txt Dockerfile.txt Producer.txt

• [x] Confluent.Kafka nuget version. (1.6.3)
• [x] Apache Kafka version. (Confluent.Kafka 1.6.3)
• [x] Client configuration. (TargetFramework as specified in the .csproj file (v4.5)
• [x] Operating system. Linux (Ubuntu)
• [ ] Provide logs (with "debug" : "..." as necessary in configuration).
• [ ] Provide broker log excerpts.
• [ ] Critical issue.

[AyeshaSadiq7]

No I didn't hear back and meanwhile found that dotnet confluent kafka is not compatible with Mono. So, I moved to another solution using MQTT broker as a bridge between Kafka server and my PMU data streaming App.

[PaskeS]

What was the fix here? Having the exact same issue which lead me to build librdkafka but this then just leads to this issue: Exception thrown: 'System.DllNotFoundException' in Confluent.Kafka.dll: 'Unable to load shared library 'centos7-librdkafka' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libcentos7-librdkafka: cannot open shared object file: No such file or directory' Stack trace:

at Confluent.Kafka.Impl.NativeMethods.NativeMethods_Centos7.rd_kafka_err2str(ErrorCode err) at Confluent.Kafka.Impl.Librdkafka.SetDelegates(Type nativeMethodsClass)

Which looks to be this caused by this -> file=libsasl2.so.3 [0]; needed by /app/bin/Debug/net5.0/runtimes/linux-x64/native/centos7-librdkafka.so [0]. Running using debian so why this should cause me to error i am clueless

[mhowlett]

@PaskeS - what version are you using? Confluent.Kafka tries to load the centos lib first in recent versions, and if it doesn't succeed attempts to load others. It looks like the fail-over isn't working for some reason. It will probably work if you install libsasl. marking this as a bug.

[PaskeS]

@mhowlett - Thanks for quick reply. I have tried with:

• Conluent.Kafka 1.6.3 (librdkafka.redist = 1.61)
• Conluent.Kafka 1.7.0-RC4 (librdkafka.redist = 1.7.0)

Running in docker - Debian 10 (aspnet:5.0) Pulled in libsasl2-2 but only goes up to libsasl2.so.2. Seems like libsasl2.so.3 is on rpm package (cyrus-sasl-lib-2.1.26-23.el7.x86_64.rpm for centOS). Tried to symlink but failed for some reason even though could see it here ( 27: trying file=/usr/lib/x86_64-linux-gnu/libsasl2.so.3)

Also still have this issue -> Invalid librdkafka version b06ff, expected at least 10502ff When using this method "RUN apt-get install -y librdkafka-dev" if you have any ideas? Not as big as the centOS issue as managed to workaround although not necessarily ideal.

[mhowlett]

RUN apt-get install -y librdkafka-dev

you shouldn't do that, suitable librdkafka binaries are bundled with Confluent.Kafka automatically via nuget. unfortunately Confluent.Kafka will use the system librdkafka on some systems and the one you have installed is not new enough.

does it work if you don't install librdkafka via apt-get?

[PaskeS]

Managed to solve this tonight, shot myself in the foot by trusting visual studio to communicate with docker. (Thought it would save me time - oh how I was wrong :() Manually running all docker/K8's commands managed to fix. @mhowlett Thank you very much for your help, sorry it was some random reason. I have no idea what the visual studio integration is doing differently but must be screwing with environment/runtime in the chain. Hopefully this will help some poor soul who makes the same mistake

[software-programmer]

HI @PaskeS

Would you be able to provide the relevant parts of your Docker file for the "manual" commands. I've followed this thread after facing the same issues, but can't obviously see what the fix is that you're discussing.

What are the manual commands?

Thanks

[amarj27]

I have the same error message. I am running an Azure Function that is consuming messages from a Kafka topic (Kafka triggered Azure Function) on a Docker container.

I tried installing these libraries in my Dockerfile based on suggestions on other threads and that helped me resolve the dll not found exception but now I am having this error System.IO.FileLoadException: Invalid librdkafka version b06ff, expected at least 10502ff -

RUN apt-get update -y RUN apt-get install -y librdkafka-dev RUN apt-get install -y libc-dev

@PaskeS @mhowlett Could you let me know if you have any suggestions on how I can resolve this?

[mhowlett]

you shouldn't install librdkafka-dev - it's unfortunately picking up the librdkafka version installed on the system rather than the version in the librdkafka.redist nuget package, which is a dependency of Confluent.Kafka.

[amarj27]

@mhowlett If I dont install this in my Dockerfile, I get a DllNotFoundException for the librdkafka dll. I am using a NodeJS Azure Function and it uses extension bundle to install these Dlls/packages.

See this thread here for reference - https://github.com/confluentinc/confluent-kafka-dotnet/issues/778#issuecomment-560225446