length error in handle_setopt

[bwlewis]

handle_setopt probably needs to use xlength. Here is a short example:

set.seed(1)
x = serialize(runif(2^29), NULL)
h = curl::new_handle()
curl::handle_setopt(h, .list = list(customrequest = "PUT"))
curl::handle_setopt(h, .list = list(post=TRUE, postfieldsize_large=length(data), postfields=x))

## Error in curl::handle_setopt(h, .list = list(post = TRUE, postfieldsize_large = length(data),  : 
##  long vectors not supported yet: ../../src/include/Rinlinedfuns.h:138

This is similar to https://github.com/r-lib/httr/issues/257. I will make a trivial pull request against handle.c that I think fixes this issue.

[jeroen]

Thanks. Are there other places in the C code where it would be safer to use Rf_xlength / R_xlen_t?

[bwlewis]

Of course, sorry I didn't check. Indeed there are a few others.

Please ignore the PR I will prepare a new one. About to get on a plane, so that will come later.

[bwlewis]

The only other place I see this potentially being a problem is in form.c's use of curl_formadd(). There is a minor rabbit hole there in that there are curl API-version specific variadic arguments and, worse, this:

This function is deprecated. Do not use! See curl_mime_init instead! (see https://curl.haxx.se/libcurl/c/curl_formadd.html).

No big deal. I prepared a PR that will follow that handles the API versioning requirements with ifdefs for now. Medium-term it looks like curl_formadd will need to be replaced.

Best,

Bryan

[jeroen]

Thanks will try to review this soon. I'm going to stick with the (old) formadd API because the mime api was only added in the last version and we need to support old versions of libcurl on old linux distributions. Curl never removes APIs so no need to migrate.

[aronatkins]

One of our users may be seeing this themselves, called from: https://github.com/rstudio/rsconnect/blob/40c162bf3c4cd9b10deed82e1df9ec1b0c107c1f/R/http-libcurl.R#L85-L90

We believe that the POST file is ~25G.

[clemenshug]

One of our users may be seeing this themselves, called from: https://github.com/rstudio/rsconnect/blob/40c162bf3c4cd9b10deed82e1df9ec1b0c107c1f/R/http-libcurl.R#L85-L90

We believe that the POST file is ~25G.

I just also ran into this issue deploying a Shiny app to RStudio Connect with a bundle size of 2.1G.

After some digging I realized I can select which library rsconnect uses for bundle uploads https://github.com/rstudio/rsconnect/blob/0c062e766890f03bfd4a7a4a62f10b43e5db6fff/R/http.R#L110.

Setting options(rsconnect.http = "curl") fixed the issue for me