Botan is not fork safe - Githubissues

bs-rscs commented 3 years ago

We recently came across an issue with Botan's "thread pool". Namely, when we use RSA operations (that utilize the thread pool) and fork the process afterwards, then the forked process will stop working correctly. In our case, it won't terminate, as it hangs in Botan::Thread_Pool::shutdown().

It looks like the forked process still has the list of threads in the thread pool, but those don't exist anymore, as the original process terminated. When trying to shut the non-existing threads down, he waits forever:

#0  futex_wait (private=<optimized out>, expected=9, 
    futex_word=0x7ff260dd9fb0 <Botan::Thread_Pool::global_instance()::g_thread_pool+80>)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:61
#1  futex_wait_simple (private=<optimized out>, expected=9, 
    futex_word=0x7ff260dd9fb0 <Botan::Thread_Pool::global_instance()::g_thread_pool+80>) at ../sysdeps/nptl/futex-internal.h:135
#2  __condvar_quiesce_and_switch_g1 (private=<optimized out>, g1index=<synthetic pointer>, wseq=<optimized out>, 
    cond=0x7ff260dd9fa0 <Botan::Thread_Pool::global_instance()::g_thread_pool+64>) at pthread_cond_common.c:416
#3  __pthread_cond_broadcast (cond=0x7ff260dd9fa0 <Botan::Thread_Pool::global_instance()::g_thread_pool+64>)
    at pthread_cond_broadcast.c:73
#4  0x00007ff25fc778fd in __gthread_cond_broadcast (__cond=<optimized out>)
    at /usr/src/debug/gcc-8.3.1-5.el8.0.2.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/x86_64-redhat-linux/bits/gthr-default.h:852
#5  std::condition_variable::notify_all (this=<optimized out>) at ../../../../../libstdc++-v3/src/c++11/condition_variable.cc:73
#6  0x00007ff260a45370 in Botan::Thread_Pool::shutdown() () from /usr/local/lib/libbotan-2.so.13
#7  0x00007ff260a45bc6 in Botan::Thread_Pool::~Thread_Pool() () from /usr/local/lib/libbotan-2.so.13
#8  0x00007ff25f298e9c in __run_exit_handlers (status=0, listp=0x7ff25f61b738 <__exit_funcs>, 
    run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
#9  0x00007ff25f298fd0 in __GI_exit (status=<optimized out>) at exit.c:139
#10 0x00007ff25f2826aa in __libc_start_main (main=0x4be3b0 <main(int, char const**)>, argc=16, argv=0x7ffcf80f5c08, 
    init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffcf80f5bf8) at ../csu/libc-start.c:342
#11 0x00000000004d214e in _start () at /usr/include/c++/8/bits/unique_ptr.h:265

The process must then be killed in order to terminate it.

We were using Botan 2.14.0 for this.

randombit commented 3 years ago

Thanks for reporting this. As a short term workaround, in your build disable module thread_utils which will remove the thread pool logic.

randombit commented 1 year ago

It's possible to disable the thread pool at runtime by setting an env variable (eg in main) and we document the problem now in 3172b53fb3e0a. Possibly we can add some lifecycle management functions to further assist but I think this issue is about as well addressed as it can be.

randombit / botan

Botan is not fork safe #2608