search

confluentinc / librdkafka

The Apache Kafka C/C++ library
Other
214 stars 3.14k forks source link

Docker Container not running with node-rdkafka and librdkafka #3691

Open abbey007 opened 2 years ago

abbey007 commented 2 years ago

Description

I am using node-rdkafka to connect my application with kafka. When i am running the application using docker i am getting error Segmentation fault (core dumped)

I have used different combination of node images to resolve the issue.

How to reproduce

Attaching my DockerFile

FROM node:14.17.3

RUN npm cache clean --force RUN apt-get update RUN apt install -y liblz4-dev RUN apt install -y libsasl2-dev RUN apt install -y libssl-dev RUN apt-get install -y ca-certificates curl gnupg \ g++ make musl-dev \ python3 unzip wget \ git

RUN git clone https://github.com/edenhill/librdkafka.git && \ cd librdkafka && \ ./configure --install-deps && \ make && \ make install

ENV BUILD_LIBRDKAFKA=0 ENV LD_LIBRARY_PATH=/usr/local/lib

RUN mkdir -p /home/node && chown -R node:node /home/node ENV NODE_ENV=local WORKDIR /home/node COPY ["package.json", "./"] RUN npm install husky -g RUN npm install -g typescript RUN npm install -g ts-node RUN npm install COPY . . EXPOSE 4003 RUN chown -R node /home/node USER node RUN rm -rf dist RUN tsc CMD node dist/bin/server.js

Checklist

Please provide the following information:

edenhill commented 2 years ago

That is likely a node-rdkafka issue, please file an issue on that repo. Also try upgrading to the latest version of librdkafka, and if still reproducible run the application in gdb and extract a backtrace (type bt in gdb) when it crashes.

abbey007 commented 2 years ago

I am reporting the issue on node-rdkafka as well, Also i am using https://github.com/edenhill/librdkafka.git which i supose using the latest version of librdkafka . I have no idea how to run the application with gdb inside a container becasue i am not able to run container due to error.

mreouven commented 10 months ago

i have the same probleme i get this error

[c] PID:1 - 11/09/2023, 8:39:02 PM LOG [Fi] Start application listening on group:find url: ["b**"] [New Thread 0x7fffde1fc700 (LWP 50168)] [New Thread 0x7fffdd7fb700 (LWP 50169)] [New Thread 0x7fffdcdfa700 (LWP 50170)] [New Thread 0x7ffd43fff700 (LWP 50171)] [New Thread 0x7ffd435fe700 (LWP 50172)] --Type for more, q to quit, c to continue without paging--bt

Thread 16 "rdk:broker-1" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffd435fe700 (LWP 50172)] 0x00007fffecff32a3 in SSL_get_peer_certificate () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1

(gdb) bt

0 0x00007fffecff32a3 in SSL_get_peer_certificate () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1

https://github.com/Blizzard/node-rdkafka/issues/1 0x00007fffed1fb409 in rd_kafka_transport_ssl_verify (rktrans=0x7ffd34005100) at rdkafka_ssl.c:582 https://github.com/Blizzard/node-rdkafka/pull/2 rd_kafka_transport_ssl_handshake (rktrans=rktrans@entry=0x7ffd34005100) at rdkafka_ssl.c:620 https://github.com/Blizzard/node-rdkafka/pull/3 0x00007fffed0c1190 in rd_kafka_transport_io_event (rktrans=rktrans@entry=0x7ffd34005100, events=events@entry=1, socket_errstr=0x0) at rdkafka_transport.c:744 https://github.com/Blizzard/node-rdkafka/issues/4 0x00007fffed0c1e6c in rd_kafka_transport_io_serve (rktrans=0x7ffd34005100, rkq=, timeout_ms=) at rdkafka_transport.c:1047 https://github.com/Blizzard/node-rdkafka/issues/5 0x00007fffed0a9fcc in rd_kafka_broker_ops_io_serve (rkb=rkb@entry=0x7ffd4c05bb50, abs_timeout=9630356814) at rdkafka_broker.c:3568 https://github.com/Blizzard/node-rdkafka/pull/6 0x00007fffed0aa4f3 in rd_kafka_broker_consumer_serve (rkb=rkb@entry=0x7ffd4c05bb50, abs_timeout=abs_timeout@entry=9630356814) at rdkafka_broker.c:4271 https://github.com/Blizzard/node-rdkafka/issues/7 0x00007fffed0aac89 in rd_kafka_broker_serve (rkb=rkb@entry=0x7ffd4c05bb50, timeout_ms=, timeout_ms@entry=1000) at rdkafka_broker.c:4413 https://github.com/Blizzard/node-rdkafka/pull/8 0x00007fffed0ab155 in rd_kafka_broker_thread_main (arg=0x7ffd4c05bb50) at rdkafka_broker.c:4561 https://github.com/Blizzard/node-rdkafka/issues/9 0x00007ffff7c7af5a in start_thread (arg=) at pthread_create.c:474 https://github.com/Blizzard/node-rdkafka/pull/10 0x00007ffff7b9aa2f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

the segmentation fault is occurring within the SSL_get_peer_certificate function from the libssl library.

milindl commented 2 months ago

I would suggest trying to use https://github.com/confluentinc/confluent-kafka-javascript which is compatible with node-rdkafka (and is based on it). It includes a precompiled version of librdkafka within which should have all the openssl dependencies built into it. I recall having to set ssl.ca.location while using node-rdkafka, maybe this is related. This isn't the case with confluent-kafka-javascript.

Example of a dockerfile using that: https://github.com/confluentinc/confluent-kafka-javascript/issues/48#issuecomment-2121819714