Closed mattjala closed 3 months ago
After a discussion with Jordan, this should probably be changed to use a pool of handles, one per server. Curl handles are made to be reused, so creating and destroying one for each request is probably wasteful.
The curl helpers now set fields on the global curl handle, instead of creating and destroying a handle for each operation.
See here: "libcurl keeps a pool of old connections alive. [...] Reusing a connection instead of creating a new one offers significant benefits in speed and required resources. [...] When you are using the easy API, or, more specifically, curl_easy_perform()
, libcurl will keep the pool associated with the specific easy handle. Then reusing the same easy handle will ensure it can reuse its connection."
Most curl requests are now done through new helper functions in
rest_vol.c
,RV_curl_post
,RV_curl_put
RV_curl_delete
, andRV_curl_get
. These functions each create a local curl handle, perform the request, then clean up the handle afterwards.In order to allow for cases where specific error codes are OK or even expected (checking for the existence of files, flush requests that expect to get HTTP 204 NO CONTENT) the curl helpers return the HTTP code that the request returned, or -1 if the request failed.
The curl helper functions take in the server's base URL through the
server_info_t
struct, so they only need the endpoint as a separate argument. For this reason, most instances ofrequest_url
have been changed torequest_endpoint
, and no longer include the base URL. Thisrequest_endpoint
is what is passed to the curl helpers.I also introduced a new global variable
H5_rest_curl_initialized_g
to track if curl is initialized. This avoids not-initializing or double-initializing it in the new helper to create a curl handle,RV_curl_setup_handle
.Places that still use the global curl handle:
H5_rest_authenticate_with_AD
since I'm not sure how to test itRV_curl_multi_perform
This is a draft PR since it's based on the branch that stores server connection information on individual file objects (#86).