Upload error to Rpubs from R markdown Error in rawToChar(response$headers) : argument 'x' must be a raw vector

[billy-s-lab]

Hi,

I am trying to publish an RMarkdown file to RPubs using rsconnect.

The file knits to HTML appropriately on my own laptop. When I try to publish the knitted HTML to RPubs, I get the following error:

Error in rawToChar(response$headers) : argument 'x' must be a raw vector

Screenshot of error below:

I have been able to publish other RMarkdown documents to RPubs in the past few days.

Any help greatly appreciated. Thank you.

sessionInfo() below:

R version 4.0.0 (2020-04-24) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS High Sierra 10.13.6

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale: [1] en_IE.UTF-8/en_IE.UTF-8/en_IE.UTF-8/C/en_IE.UTF-8/en_IE.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] RColorBrewer_1.1-2 pheatmap_1.0.12 factoextra_1.0.7 FactoMineR_2.3 gridExtra_2.3
[6] mgcv_1.8-31 nlme_3.1-147 gpairs_1.3.3 scales_1.1.1 kableExtra_1.1.0
[11] knitr_1.28 cmprsk_2.2-9 cowplot_1.0.0 survminer_0.4.6 survival_3.1-12
[16] lmerTest_3.1-2 wesanderson_0.3.6 PredictABEL_1.2-4 DescTools_0.99.35 lmtest_0.9-37
[21] zoo_1.8-8 lme4_1.1-23 Matrix_1.2-18 MASS_7.3-51.6 ggfortify_0.4.10
[26] GGally_1.5.0 ggrepel_0.8.2 ggsci_2.9 rstatix_0.6.0 ggpubr_0.3.0
[31] broom_0.5.6 forcats_0.5.0 stringr_1.4.0 dplyr_0.8.5 purrr_0.3.4
[36] readr_1.3.1 tidyr_1.1.0 tibble_3.0.1 ggplot2_3.3.0 tidyverse_1.3.0
[41] readxl_1.3.1 rsconnect_0.8.16

loaded via a namespace (and not attached): [1] minqa_1.2.4 colorspace_1.4-1 ggsignif_0.6.0 ellipsis_0.3.1 rio_0.5.16
[6] htmlTable_1.13.3 base64enc_0.1-3 fs_1.4.1 rstudioapi_0.11 farver_2.0.3
[11] PBSmodelling_2.68.8 fansi_0.4.1 mvtnorm_1.1-0 lubridate_1.7.8 xml2_1.3.2
[16] leaps_3.1 splines_4.0.0 Formula_1.2-3 jsonlite_1.6.1 nloptr_1.2.2.1
[21] km.ci_0.5-2 cluster_2.1.0 dbplyr_1.4.3 png_0.1-7 compiler_4.0.0
[26] httr_1.4.1 backports_1.1.7 assertthat_0.2.1 cli_2.0.2 acepack_1.4.1
[31] htmltools_0.4.0 tools_4.0.0 gtable_0.3.0 glue_1.4.1 Rcpp_1.0.4.6
[36] carData_3.0-3 cellranger_1.1.0 vctrs_0.3.0 xfun_0.14 openxlsx_4.1.5
[41] rvest_0.3.5 lifecycle_0.2.0 statmod_1.4.34 XML_3.99-0.3 barcode_1.1
[46] hms_0.5.3 expm_0.999-4 curl_4.3 KMsurv_0.1-5 rpart_4.1-15
[51] reshape_0.8.8 latticeExtra_0.6-29 stringi_1.4.6 checkmate_2.0.0 boot_1.3-25
[56] zip_2.0.4 rlang_0.4.6 pkgconfig_2.0.3 evaluate_0.14 lattice_0.20-41
[61] ROCR_1.0-11 labeling_0.3 htmlwidgets_1.5.1 tidyselect_1.1.0 plyr_1.8.6
[66] magrittr_1.5 R6_2.4.1 generics_0.0.2 Hmisc_4.4-0 DBI_1.1.0
[71] pillar_1.4.4 haven_2.2.0 foreign_0.8-79 withr_2.2.0 scatterplot3d_0.3-41 [76] abind_1.4-5 nnet_7.3-14 modelr_0.1.8 crayon_1.3.4 car_3.0-7
[81] survMisc_0.5.5 rmarkdown_2.3 jpeg_0.1-8.1 grid_4.0.0 data.table_1.12.8
[86] flashClust_1.01-2 vcd_1.4-7 webshot_0.5.2 reprex_0.3.0 digest_0.6.25
[91] xtable_1.8-4 numDeriv_2016.8-1.1 munsell_0.5.0 viridisLite_0.3.0 tcltk_4.0.0
[96] fortunes_1.5-4

[jmcphers]

I have been able to publish other RMarkdown documents to RPubs

Since other documents are publishing without issue, we probably won't be able to figure out what's going on without looking at the actual doc you're trying to publish. Are you able to share that?

[bpbond]

I'm also experiencing this error, and would welcome any insight into what's causing it.

My RMarkdown file is here, and session information is below. Thanks!

R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] emo_0.0.0.9000     ggplot2_3.3.3      RColorBrewer_1.1-2

loaded via a namespace (and not attached):
 [1] tidyselect_1.1.0  xfun_0.19         remotes_2.2.0     purrr_0.3.4      
 [5] colorspace_2.0-0  vctrs_0.3.6       generics_0.1.0    testthat_3.0.1   
 [9] htmltools_0.5.0   usethis_2.0.0     viridisLite_0.3.0 yaml_2.2.1       
[13] utf8_1.1.4        rlang_0.4.10      pkgbuild_1.2.0    pillar_1.4.7     
[17] glue_1.4.2        withr_2.3.0       sessioninfo_1.1.1 lifecycle_0.2.0  
[21] stringr_1.4.0     munsell_0.5.0     gtable_0.3.0      devtools_2.2.1   
[25] evaluate_0.14     memoise_1.1.0     labeling_0.4.2    knitr_1.30       
[29] callr_3.5.1       ps_1.5.0          curl_4.3          fansi_0.4.1      
[33] Rcpp_1.0.5        scales_1.1.1      desc_1.2.0        pkgload_1.1.0    
[37] farver_2.0.3      fs_1.5.0          digest_0.6.27     stringi_1.5.3    
[41] processx_3.4.5    dplyr_1.0.2       grid_3.6.1        rprojroot_2.0.2  
[45] cli_2.2.0         tools_3.6.1       magrittr_2.0.1    tibble_3.0.4     
[49] crayon_1.3.4      pkgconfig_2.0.3   ellipsis_0.3.1    rsconnect_0.8.16 
[53] prettyunits_1.1.1 lubridate_1.7.9.2 assertthat_0.2.1  rmarkdown_2.5    
[57] rstudioapi_0.13   R6_2.5.0          compiler_3.6.1  

[bpbond]

Another data point: my colleague @stephpenn1 is running R 4.03 and tried to publish this same Rmd file:

Error in curl::curl_fetch_memory(url, handle = handle) : 
  HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)
Calls: <Anonymous> ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>

[aronatkins]

This is also reported as a curl issue here: https://github.com/jeroen/curl/issues/232

I have been able to recreate this problem using the Rmd at https://github.com/JGCRI/Rworkshops/blob/master/ggplot2-basics.Rmd. The issue appears to be related to the size of the generated content - anything smaller than ~16M uploads successfully.

With that ggplot2-basics.Rmd, I can successfully deploy by removing everything below the ## 6. The ggplot call heading. Not a workaround or solution, but maybe a lead.

[aronatkins]

The code in question, where we set some large-file parameters and then send the rpubs request: https://github.com/rstudio/rsconnect/blob/8c11a55bbbe111a4df1e37ddaeb6f19a86f32d39/R/http-libcurl.R#L81-L128

[bpbond]

Ah, thanks @aronatkins ! Hopefully this is useful for you guys, but for my case, I should be able to reduce the size of the content and thus upload.

[aronatkins]

More information: If I disable HTTP/2 with the following code inserted into httpLibCurl:

  # disable http/2, per https://cran.r-project.org/web/packages/curl/vignettes/intro.html#disabling_http2
  curl::handle_setopt(handle, http_version = 2)

I get the original error reported in this issue:

Error in rawToChar(response$headers) : argument 'x' must be a raw vector

Running through in the debugger, this is because response is NULL. The response is NULL because we are falling into the block here:

https://github.com/rstudio/rsconnect/blob/8c11a55bbbe111a4df1e37ddaeb6f19a86f32d39/R/http-libcurl.R#L131-L133

The e error object:

<simpleError in curl::curl_fetch_memory(url, handle = handle): transfer closed with outstanding read data remaining>

@jmcphers - It feels as if we have at least one error path in httpLibCurl that is not trapped, but underlying cause may be something about the rpubs server response.

[jeroen]

What httpd is rpubs running? transfer closed with outstanding read data remaining could mean that your server is setting an invalid content-length on the response headers. Which could happen for example when you are mixing up gzipped and non-gzipped responses in your cache.

[aronatkins]

I think I've found that the rpubs server is erring requests that send too-large uploads, and does so without reading the full request body. That feels OK, so we may simply need to react to those transfer errors in rsconnect (possibly preventing them when we know the upload is too large).

[hadley]

This Rmd is about ~17 Mb on my computer:

---
output: html_document
---

```{r}
library(ggplot2)

plot <- ggplot(diamonds, aes(carat, price)) + geom_point()
for (i in 1:80) {
  print(plot)
}

And I published it with:

```R
rpubsUpload("large-file", "~/Desktop/ggplot2.html", "~/Desktop/ggplot2.Rmd")