Calls `curl_multi_assign` with null

[bevanjkay]

When building the latest version of curl at Homebrew, we ran into a segfault when regression testing Julia as a curl dependent. The segfault issue has been resolved in curl, however it likely points to an issue in the curl multi handler in Julia.

See curl comment for reference; https://github.com/curl/curl/issues/14860#issuecomment-2343504695

For our CI build; Complete logs are available here. The referenced error starts here on macOS 14 arm64.

[an-Iceberg]

I am affected by this issue as well. I can't update any packages i've installed because a segmentation fault at curl_multi_assign happens.

[giordano]

@bevanjkay do you have a self-contained reproducer?

[giordano]

If I change

diff --git a/src/Curl/Curl.jl b/src/Curl/Curl.jl
index fd9b4c9..a2cc146 100644
--- a/src/Curl/Curl.jl
+++ b/src/Curl/Curl.jl
@@ -51,6 +51,7 @@ function curl_multi_socket_action(multi_handle, s, ev_bitmask, running_handles)
     ccall((:curl_multi_socket_action, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Cint, Ptr{Cint}), multi_handle, s, ev_bitmask, running_handles)
 end
 function curl_multi_assign(multi_handle, sockfd, sockp)
+    multi_handle == C_NULL && error("multi_handle is NULL")
     ccall((:curl_multi_assign, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Ptr{Cvoid}), multi_handle, sockfd, sockp)
 end

all tests pass for me. Please, provide a self-contained minimal reproducer, otherwise it looks complicated to address this issue.

[giordano]

And with

diff --git a/src/Curl/Curl.jl b/src/Curl/Curl.jl
index fd9b4c9..546ef21 100644
--- a/src/Curl/Curl.jl
+++ b/src/Curl/Curl.jl
@@ -51,6 +51,8 @@ function curl_multi_socket_action(multi_handle, s, ev_bitmask, running_handles)
     ccall((:curl_multi_socket_action, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Cint, Ptr{Cint}), multi_handle, s, ev_bitmask, running_handles)
 end
 function curl_multi_assign(multi_handle, sockfd, sockp)
+    @info "Hello from curl_multi_assign"
+    multi_handle == C_NULL && error("multi_handle is NULL")
     ccall((:curl_multi_assign, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Ptr{Cvoid}), multi_handle, sockfd, sockp)
 end

I tried to follow the (non-minimal) reproducer by @carlocab in https://github.com/curl/curl/issues/14860#issue-2519438579 and I still can't see multi_handle being NULL:

% julia -p 3 --project=/tmp -e 'using Pkg; Pkg.add("Example")'
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
   Resolving package versions...
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
   Installed Example ─ v0.5.3
    Updating `/private/tmp/Project.toml`
  [7876af07] + Example v0.5.3
    Updating `/private/tmp/Manifest.toml`
  [7876af07] + Example v0.5.3
Precompiling packages finished.
  1 dependency successfully precompiled in 1 seconds. 25 already precompiled.

Note that Example is being installed (I had deleted the source from my ${JULIA_DEPOT_PATH}/packages directory), and the message Hello from curl_multi_assign to confirm my custom Downloads version is being used.

[giordano]

Alright, I finally managed to hit a NULL handle with

% julia --project=/tmp -e 'using Downloads; Downloads.download("https://example.org", IOBuffer())'
┌ Error: socket_callback: unexpected error
│   err = multi.handle is NULL
└ @ Downloads.Curl ~/repo/Downloads.jl/src/Curl/Multi.jl:204

but only with libcurl 8.10.0+. This is happening at the exit of julia, so probably in the finalizer of the https://github.com/JuliaLang/Downloads.jl/blob/89d3c7dded535a77551e763a437a6d31e4d9bf84/src/Curl/Curl.jl#L93-L101

All my tests above were with Julia master (0073917331) and built-in libcurl 8.6.0. I also can't reproduce it with a local build of libcurl 8.9.0. What I did here was to use same version of Julia and Downloads.jl and link it to a local build of libcurl 8.10.0 or 8.10.1. But the thing is that the code path on the julia side changed only by changing libcurl version.