double free or corruption (fasttop) when creating multiple contexts from different threads

[shdown]

Hello,

we have a program that creates multiple LWS contexts in different threads. It worked perfectly fine on other our servers, but on a new server we have the following issue:

*** Error in `/home/dumper/worker': double free or corruption (fasttop): 0x00007f10e8006080 ***

We collected the core file and inspected it with gdb, backtrack says it crashed in

[Current thread is 1 (Thread 0x7f1109081700 (LWP 7868))]
(gdb) bt
#0  0x00007f111e7ce438 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007f111e7d003a in __GI_abort () at abort.c:89
#2  0x00007f111e8107fa in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f111e929fd8 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007f111e81938a in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>, str=0x7f111e92a0a0 "double free or corruption (fasttop)", action=3) at malloc.c:5020
#4  _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3874
#5  0x00007f111e81d58c in __GI___libc_free (mem=<optimized out>) at malloc.c:2975
#6  0x00007f111e14f0cd in CRYPTO_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#7  0x00007f111e151c7c in OBJ_NAME_add () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#8  0x00007f111e211835 in EVP_add_cipher () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#9  0x00007f111e2159d1 in OpenSSL_add_all_ciphers () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#10 0x00007f111e2159ae in OPENSSL_add_all_algorithms_noconf () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#11 0x00007f111f51526e in lws_context_init_ssl_library () from /usr/local/lib/libwebsockets.so.17
#12 0x00007f111f51b19c in lws_create_context () from /usr/local/lib/libwebsockets.so.17
#13 0x000000000040b1a2 in weso_run (weso=weso@entry=0x7f10ec0008c0, cb_connected=cb_connected@entry=0x40f250 <ws_cb_connected>, cb_msg=cb_msg@entry=0x40d560 <ws_cb_msg>, ud=ud@entry=0x40000)
    at ./weso.c:502
#14 0x000000000040f03e in ws_thread_func (ud=<optimized out>) at worker.c:1051
#15 0x00007f111eb6a6ba in start_thread (arg=0x7f1109081700) at pthread_create.c:333
#16 0x00007f111e8a051d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

So, what are we doing wrong? Any help would be appreciated.

[lws-team]

This is to do with your OpenSSL blowing chunks on being reinitialized. You can try leaving the context flag that tells it to initialize tls off on the second creation.

[shdown]

You can try leaving the context flag that tells it to initialize tls off on the second creation.

I don’t understand this part. What exactly (in terms of code) are you suggesting?

[lws-team]

info.options = LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT;

[shdown]

We already set this flag each time we call lws_create_context.

[lws-team]

Yeah. It makes lws ask the tls library to initialize itself.

You can try leaving the context flag that tells it to initialize tls off on the second creation.

ie, try NOT telling it to reinit the tls library each time, just the first context.