search

Azure / azure-storage-cpplite

Lite version of C++ Client Library for Microsoft Azure Storage
MIT License
25 stars 44 forks source link

CurlEasyClient is not thread-safe #124

Open morten-ofstad opened 3 years ago

morten-ofstad commented 3 years ago

The CurlEasyClient class calls curl_global_init/curl_global_cleanup in its constructor/destructor. These functions increase/decrease an initialized count, but when called from multiple threads this breaks since it's not using an atomic increment/decrement (the initialized count gets read and increased by two threads and then written with the new value which is only increased by one, or similar in the destructor). This lead to crashes when running one of our application for several hours because curl would suddenly be deinitialized when it shouldn't be.

We've had to patch azure-storage-cpplite to remove the calls to curl_global_init/curl_global_cleanup from the CurlEasyClient class, but this change could possibly break other applications that depend on automatic cleanup when using azure-storage-cpplite in a single-threaded application so I'm not sure how you would want to solve this. curl_easy_init will automatically call curl_global_init if it hasn't already been called, but in that case it won't clean up on exit (see https://github.com/curl/curl/blob/master/lib/easy.c).

Jinming-Hu commented 3 years ago

Hi @morten-ofstad , thanks for reporting this bug.

However, this cpplite sdk has been deprecated and is now in maintenance mode. Can you try our next generation of Storage SDK?

morten-ofstad commented 3 years ago

This bug was found in a deployment for a large customer. We're evaluating switching to the next generation SDK for the next version of our software. We're using azure-storage-cpplite only to sign requests, so we just create a request, sign it and read back the HTTP-headers -- then we use our HTTP code which uses curl-multi to perform the requests asynchronously and in parallel for max. throughput. This might not be the place for it, but do you have any advice on how to best use the next gen Azure SDK for only signing requests?

Jinming-Hu commented 3 years ago

We're using azure-storage-cpplite only to sign requests, so we just create a request, sign it and read back the HTTP-headers -- then we use our HTTP code which uses curl-multi to perform the requests asynchronously and in parallel for max. throughput. This might not be the place for it, but do you have any advice on how to best use the next gen Azure SDK for only signing requests?

Yes, next gen sdk was designed to be able to handle this scenario. Please open an issue in Track2 repo to further discuss.