curl failure with Apple M Series and macOS Sonoma 14.5

[adamhsparks]

I've run into a spot of trouble just this week when I try to reknit my vignettes for {weatherOz} that are precompiled to reduce undue CRAN stresses due to needing Internet access.

This week when I've tried to build the Rmd files, on both my M1 Air and my M3 MacBook Pro, the process stops with one of the internal operations that uses {curl} to fetch some data.

Here's a stack trace, thanks to @paleolimbot for the assist with getting this via rOpenSci's Slack group.

If I run the offending curl::curl_download() directly from the R prompt, it works for me. I can put it in an Rmd file and it renders, but when I try to knit as follows, it stops.

$ R -d lldb 
$ run
> knit(input = "vignettes/weatherOz.Rmd.orig",
+      output = "vignettes/weatherOz.Rmd")

processing file: vignettes/weatherOz.Rmd.orig
  |.......................                 |  57% [nearby-toowoomba]           Process 85419 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x44206f4720656864)
    frame #0: 0x00000001a6f4c4fc libcurl.4.dylib`Curl_bufq_is_empty + 8
libcurl.4.dylib`Curl_bufq_is_empty:
->  0x1a6f4c4fc <+8>:  ldp    x9, x8, [x8, #0x10]
    0x1a6f4c500 <+12>: cmp    x9, x8
    0x1a6f4c504 <+16>: cset   w0, hs
    0x1a6f4c508 <+20>: ret    
Target 0: (R) stopped.

This is the function where the failure occurs.

https://github.com/ropensci/weatherOz/blob/20c56de1b6488284b705ee80fefd1eb6f581b6ed/R/get_bom_metadata.R#L11

[adamhsparks]

I've had the same issue now trying to use devtools::check_win_devel() from a regular R session in my terminal, R segfaulted when the package was to be uploaded.

[jeroen]

Did this start happening after you upgraded to Sonoma 14.5 ? What is the curl_version() on your system?

Could you test if the problem goes away if you install the curl package from source, and then restart R?

install.packages("curl", type = "source")

[adamhsparks]

Yes, AFAIK it's related to the upgrade to 14.5 but I can't prove it.

I've tried building from source as well using the system version and the Homebrew version of curl to build from source but get the same result. That was the first thing I tried.

Here's the Homebrew version that's installed.

> curl -V
curl 8.7.1 (aarch64-apple-darwin23.4.0) libcurl/8.7.1 (SecureTransport) OpenSSL/3.3.0 zlib/1.2.12 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libssh2/1.11.0 nghttp2/1.61.0 librtmp/2.3 OpenLDAP/2.6.7
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

Here's what's reported for the system curl version that's installed.

❯ curl -V
curl 8.6.0 (x86_64-apple-darwin23.0) libcurl/8.6.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.61.0
Release-Date: 2024-01-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

[lcolladotor]

Hi,

I just re-opened https://github.com/Bioconductor/BiocFileCache/issues/48 which I had linked at https://github.com/jeroen/curl/issues/232#issuecomment-1677729043, and was initially resolved by upgrading my macOS from Ventura 13.4 to 13.5.

As @adamhsparks, I also just updated my Apple M1 Max to macOS Sonoma 14.5 and started running into issues.

If it helps, here's the system version of curl installed.

 % curl -V
curl 8.6.0 (x86_64-apple-darwin23.0) libcurl/8.6.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.61.0
Release-Date: 2024-01-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

As I reported at https://github.com/Bioconductor/BiocFileCache/issues/48#issuecomment-2120473001, I've also tried it with R 4.4.0 and install.packages("curl", type = "source").

At https://github.com/Bioconductor/BiocFileCache/issues/48#issuecomment-2120487236, I show that with utils::download.file(), things do work. Also curl::curl_download() and httr::GET() do seem to work as noted at https://github.com/Bioconductor/BiocFileCache/issues/48#issuecomment-2120527682 and https://github.com/Bioconductor/BiocFileCache/issues/48#issuecomment-2120540726, respectively.

I posted this info on the odd chance that our issues were related and/or help identify the source of issues.

Best, Leo

[lcolladotor]

Hi again,

Thanks to @mtmorgan at https://github.com/Bioconductor/BiocFileCache/issues/48#issuecomment-2120806580, I got to a smaller reproducible example for my case just using httr::HEAD(). As noted on that other thread, httr::HEAD() does work on a linux OS machine I have access to.

Here's the traceback output:

> httr::HEAD("http://www.dropbox.com/s/srkb2ife75px2yz/modeling_results_BayesSpace_k09.Rdata?dl=1")
Error in curl::curl_fetch_memory(url, handle = handle) :
  HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
> traceback()
5: curl::curl_fetch_memory(url, handle = handle)
4: request_fetch.write_memory(req$output, req$url, handle)
3: request_fetch(req$output, req$url, handle)
2: request_perform(req, hu$handle$handle)
1: httr::HEAD("http://www.dropbox.com/s/srkb2ife75px2yz/modeling_results_BayesSpace_k09.Rdata?dl=1")

Here's the longer reprex::reprex() output on macOS Sonoma 14.5:

httr::HEAD("http://www.dropbox.com/s/srkb2ife75px2yz/modeling_results_BayesSpace_k09.Rdata?dl=1")
#> Error in curl::curl_fetch_memory(url, handle = handle): HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)

options(width = 120)
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.4.0 (2024-04-24)
#>  os       macOS Sonoma 14.5
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       America/New_York
#>  date     2024-05-20
#>  pandoc   3.1.12.1 @ /opt/homebrew/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  cli           3.6.2   2023-12-11 [1] CRAN (R 4.4.0)
#>  curl          5.2.1   2024-03-01 [1] CRAN (R 4.4.0)
#>  digest        0.6.35  2024-03-11 [1] CRAN (R 4.4.0)
#>  evaluate      0.23    2023-11-01 [1] CRAN (R 4.4.0)
#>  fastmap       1.2.0   2024-05-15 [1] CRAN (R 4.4.0)
#>  fs            1.6.4   2024-04-25 [1] CRAN (R 4.4.0)
#>  glue          1.7.0   2024-01-09 [1] CRAN (R 4.4.0)
#>  htmltools     0.5.8.1 2024-04-04 [1] CRAN (R 4.4.0)
#>  httr          1.4.7   2023-08-15 [1] CRAN (R 4.4.0)
#>  knitr         1.46    2024-04-06 [1] CRAN (R 4.4.0)
#>  lifecycle     1.0.4   2023-11-07 [1] CRAN (R 4.4.0)
#>  magrittr      2.0.3   2022-03-30 [1] CRAN (R 4.4.0)
#>  purrr         1.0.2   2023-08-10 [1] CRAN (R 4.4.0)
#>  R.cache       0.16.0  2022-07-21 [1] CRAN (R 4.4.0)
#>  R.methodsS3   1.8.2   2022-06-13 [1] CRAN (R 4.4.0)
#>  R.oo          1.26.0  2024-01-24 [1] CRAN (R 4.4.0)
#>  R.utils       2.12.3  2023-11-18 [1] CRAN (R 4.4.0)
#>  R6            2.5.1   2021-08-19 [1] CRAN (R 4.4.0)
#>  reprex        2.1.0   2024-01-11 [1] CRAN (R 4.4.0)
#>  rlang         1.1.3   2024-01-10 [1] CRAN (R 4.4.0)
#>  rmarkdown     2.27    2024-05-17 [1] CRAN (R 4.4.0)
#>  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.4.0)
#>  styler        1.10.3  2024-04-07 [1] CRAN (R 4.4.0)
#>  vctrs         0.6.5   2023-12-01 [1] CRAN (R 4.4.0)
#>  withr         3.0.0   2024-01-16 [1] CRAN (R 4.4.0)
#>  xfun          0.44    2024-05-15 [1] CRAN (R 4.4.0)
#>  yaml          2.3.8   2023-12-11 [1] CRAN (R 4.4.0)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

curl::curl_version()
#> $version
#> [1] "8.6.0"
#> 
#> $ssl_version
#> [1] "(SecureTransport) LibreSSL/3.3.6"
#> 
#> $libz_version
#> [1] "1.2.12"
#> 
#> $libssh_version
#> [1] NA
#> 
#> $libidn_version
#> [1] NA
#> 
#> $host
#> [1] "x86_64-apple-darwin23.0"
#> 
#> $protocols
#>  [1] "dict"    "file"    "ftp"     "ftps"    "gopher"  "gophers" "http"    "https"   "imap"    "imaps"   "ldap"   
#> [12] "ldaps"   "mqtt"    "pop3"    "pop3s"   "rtsp"    "smb"     "smbs"    "smtp"    "smtps"   "telnet"  "tftp"   
#> 
#> $ipv6
#> [1] TRUE
#> 
#> $http2
#> [1] TRUE
#> 
#> $idn
#> [1] FALSE

^{Created on 2024-05-20 with reprex v2.1.0}

Best, Leo

[jeroen]

@lcolladotor I think these may be 2 different issues in recent libcurl actually 😞 . Thanks for the reproducer, that will make it easier to diagnose your problem.

[lcolladotor]

I see 😔. Well, it was worth a shot. Thanks for posting https://github.com/curl/curl/issues/13725 already (which is the root issue for the problem I reported).

Good luck tracing the issue Adam reported!

And thanks again for maintaining curl and httr and all these great packages!!!! ^^

[jeroen]

Supposedly, both issues are resolved in libcurl 8.8.0.

[jeroen]

@adamhsparks IIUC your bug is triggered only via Scott's crul package. You can probably work around it by switching to httr2 or curl to do http requests.

[lcolladotor]

Thanks @jeroen!

I've confirmed at https://github.com/Bioconductor/BiocFileCache/issues/48#issuecomment-2124935008 that libcurl 8.8.0 resolves the issue I reported.

[adamhsparks]

Weird. I managed to get the same sort of error using devtools::release() when I tried to upload a package to CRAN the other day (not {weatherOz}) and I can't duplicate this with {nasapower}, which uses {crul} but it happens at different points with {weatherOz}.

I'm really not wanting to refactor the whole codebase to change the HTTP client. 😔

Gotta do what you gotta do. Thanks for looking into it.

[adamhsparks]

I fixed this issue. I replaced any calls to curl::curl_download() with utils::download.file() and had one curl::curl_fetch_memory(), which I refactored to use utils::download.file(). These occurrences were getting data from an FTP server where it was segfaulting when using {curl}. It's all happy now as far as I can tell and I don't have any issues with {crul}, i.e., I didn't make any changes to API calls and it's fine.