search

cloudyr / googleCloudStorageR

Google Cloud Storage API to R
https://code.markedmondson.me/googleCloudStorageR
Other
104 stars 29 forks source link

http2 framing layer error #71

Closed GilianPonte closed 6 years ago

GilianPonte commented 7 years ago 
library(googleCloudStorageR)
library("googleAuthR")

## get object info in the default bucket
gcs_auth()
gcs_global_bucket("dashboards-nextail")
bucket <- gcs_get_global_bucket()
objects <- gcs_list_objects(bucket)

## upload
upload_try <- googleCloudStorageR::gcs_upload(file = "C:/Users/GilPon/Documents/Periode_week_en_dag_dashboard_tm_20170529.xlsx", bucket = "dashboards-nextail", name = "Periode_week_en_dag_dashboard_tm_20170529.xlsx", upload_type = "resumable")

Running this code gives me an error:


Error in curl::curl_fetch_memory(url, handle = handle) :
  Stream error in the HTTP/2 framing layer

I have done some googeling but im out of options. Might be an unclosed connection or a connection that gets interuppted. I don't know..

MarkEdmondson1234 commented 7 years ago

Hmm similar errors found here: https://github.com/jennybc/googlesheets/issues/298 https://github.com/jennybc/googlesheets/issues/312 https://stackoverflow.com/questions/42495967/stream-error-in-the-http-2-framing-layer-spotify-api-in-r-script

Perhaps a Google server side setting? Weird thing is it works fine for me - could you run sessionInfo() to see if its OS related? Mine is:

> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] googleCloudStorageR_0.2.0.9000

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.11           crayon_1.3.2           digest_0.6.12          withr_1.0.2           
 [5] mime_0.5               R6_2.2.1               jsonlite_1.4           xtable_1.8-2          
 [9] magrittr_1.5           httr_1.2.1             rlang_0.1.1.9000       curl_2.6              
[13] testthat_1.0.2.9000    googleAuthR_0.5.1.9000 devtools_1.13.1.9000   tools_3.3.1           
[17] shiny_1.0.3.9000       httpuv_1.3.3           pkgload_0.0.0.9000     pkgbuild_0.0.0.9000   
[21] memoise_1.1.0          htmltools_0.3.6        openssl_0.9.6
MarkEdmondson1234 commented 7 years ago

Also please run 

> curl::curl_version()
$version
[1] "7.43.0"

$ssl_version
[1] "SecureTransport"

$libz_version
[1] "1.2.5"

$libssh_version
[1] NA

$libidn_version
[1] NA

$host
[1] "x86_64-apple-darwin15.0"

$protocols
 [1] "dict"   "file"   "ftp"    "ftps"   "gopher" "http"   "https"  "imap"   "imaps"  "ldap"   "ldaps"  "pop3"   "pop3s"  "rtsp"  
[15] "smb"    "smbs"   "smtp"   "smtps"  "telnet" "tftp"  

$ipv6
[1] TRUE

$http2
[1] FALSE
MarkEdmondson1234 commented 7 years ago

@jeroen a similar problem to some other threads. These are calls directly to Google servers.

GilianPonte commented 7 years ago

SessionInfo()

R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=Dutch_Netherlands.1252  LC_CTYPE=Dutch_Netherlands.1252    LC_MONETARY=Dutch_Netherlands.1252 LC_NUMERIC=C                      
[5] LC_TIME=Dutch_Netherlands.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] googleAuthR_0.5.1.9000    googleCloudStorageR_0.2.0

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.8     digest_0.6.11   crayon_1.3.2    withr_1.0.2     mime_0.5        R6_2.2.0        jsonlite_1.3    xtable_1.8-2    magrittr_1.5    httr_1.2.1     
[11] curl_2.3        testthat_1.0.2  devtools_1.12.0 tools_3.3.2     shiny_1.0.1     httpuv_1.3.3    memoise_1.0.0   htmltools_0.3.5 openssl_0.9.5  
Warning message:
closing unused connection 3 (C:\Users\GilPon\Documents\Periode_week_en_dag_dashboard_tm_20170529.xlsx)

curl::curl_version()

$version
[1] "7.51.0"

$ssl_version
[1] "OpenSSL/1.0.2j"

$libz_version
[1] "1.2.8"

$libssh_version
[1] "libssh2/1.8.0"

$libidn_version
[1] NA

$host
[1] "x86_64-w64-mingw32"

$protocols
 [1] "dict"   "file"   "ftp"    "ftps"   "gopher" "http"   "https"  "imap"   "imaps"  "ldap"   "ldaps"  "pop3"   "pop3s"  "rtmp"   "rtsp"   "scp"    "sftp"  
[18] "smtp"   "smtps"  "telnet" "tftp"  

$ipv6
[1] TRUE

$http2
[1] TRUE

notice the: 

Warning message:
closing unused connection 3 (C:\Users\GilPon\Documents\Periode_week_en_dag_dashboard_tm_20170529.xlsx)

at session info... Uploading another file gives the same error.

MarkEdmondson1234 commented 7 years ago

Please update to the latest version of curl, 2.6, restart your R session and try again

install.packages("curl")
jeroen commented 7 years ago

This means that either the server or your company proxy server is breaking the http2 connection.

This is a server bug, but you can work around it by disabling http2 in the curl handles, for example like here: https://github.com/ropensci/monkeylearn/pull/15

MarkEdmondson1234 commented 7 years ago

Thanks @jeroen ! I will report it to Google too.

MarkEdmondson1234 commented 6 years ago

If this comes up again, run this line to fix it:

httr::set_config(httr::config(http_version = 0))

Won't put it in the package yet as its a Google server side bug and it may be fixed in future, and pretty rare I think.

MarkEdmondson1234 commented 6 years ago

This is now set in googleAuthRv 0.6.0 so should be fixed now

AvER84 commented 11 months ago

Hi I get this problem using bigrquery and dplyr now and it is really crippling. Would it be possible to have an option in bigrquery to disable http2? Im developing shiny apps for use local on my machine and on shinyapps.io, so solutions around what curl my machine is using maybe aren't gonna be a whole fix for me :/. httr::set_config(httr::config(http_version = 0)) doesnt help

AvER84 commented 11 months ago

sessionInfo() R version 4.3.1 (2023-06-16) Platform: x86_64-apple-darwin20 (64-bit) Running under: macOS Monterey 12.0.1

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib; LAPACK version 3.11.0

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/London tzcode source: internal

attached base packages: [1] stats graphics grDevices utils datasets methods
[7] base

other attached packages: [1] bigrquery_1.4.2.9000

loaded via a namespace (and not attached): [1] vctrs_0.6.4 httr_1.4.7 cli_3.6.1
[4] rlang_1.1.1 DBI_1.1.3 generics_0.1.3
[7] assertthat_0.2.1 jsonlite_1.8.7 glue_1.6.2
[10] bit_4.0.5 gargle_1.5.2 colorspace_2.1-0 [13] dbplyr_2.3.4 scales_1.2.1 fansi_1.0.5
[16] grid_4.3.1 munsell_0.5.0 tibble_3.2.1
[19] lifecycle_1.0.3 compiler_4.3.1 dplyr_1.1.3
[22] fs_1.6.3 Rcpp_1.0.11 pkgconfig_2.0.3
[25] rstudioapi_0.15.0 R6_2.5.1 tidyselect_1.2.0 [28] utf8_1.2.4 pillar_1.9.0 magrittr_2.0.3
[31] gtable_0.3.4 tools_4.3.1 bit64_4.0.5
[34] ggplot2_3.4.4

AvER84 commented 11 months ago

$version [1] "7.77.0"

$ssl_version [1] "(SecureTransport) LibreSSL/2.8.3"

$libz_version [1] "1.2.11"

$libssh_version [1] NA

$libidn_version [1] NA

$host [1] "x86_64-apple-darwin21.0"

$protocols [1] "dict" "file" "ftp" "ftps" "gopher" "gophers" "http" "https" "imap"
[10] "imaps" "ldap" "ldaps" "mqtt" "pop3" "pop3s" "rtsp" "smb" "smbs"
[19] "smtp" "smtps" "telnet" "tftp"

$ipv6 [1] TRUE

$http2 [1] TRUE

$idn [1] FALSE

AvER84 commented 11 months ago

of course times have changed new option needed and i feel silly.

solved with:

httr::set_config(httr::config(http_version = 1))

dsaada commented 6 months ago

@AvER84 where did you put the code: httr::set_config(httr::config(http_version = 1))

Because on my side, I use GoogleCloudStorage API,

into a loop, I use the function gcs_upload, I have a loop of 2500 lines, but with a bug arround loop index 1600.

I tru to put httr::set_config(httr::config(http_version = 1)),

but this is not working.

Do you have some ideas?

Many thanks

David