cloudyr / googleCloudStorageR

Google Cloud Storage API to R
https://code.markedmondson.me/googleCloudStorageR
Other
104 stars 29 forks source link

Auto refresh of service account credentials fails behind reverse proxy - Error in curl::curl_fetch_disk(url, x$path, handle = handle) : gnutls_handshake() failed: An unexpected TLS packet was received. #155

Closed RightChain closed 2 years ago

RightChain commented 2 years ago

I have a dockerized shiny application using the googleCloudStorageR package for loading and saving data files. I am able to successfully (without issues) run the application from my local machine as R code, within a docker container from my local machine, and through the ShinyProxy.io service on my local machine. When I run my code on my production server behind a reverse proxy/load balancer (Google Load Balancer) I experience issues. The package successfully authenticates from a service account JSON on load but fails to renew its credentials after an hour. These are the main errors when running the code with debug lvl 2 enabled

Error in curl::curl_fetch_disk(url, x$path, handle = handle) : gnutls_handshake() failed: An unexpected TLS packet was received.

and

Warning: Error in : Request failed before finding status code: gnutls_handshake() failed: An unexpected TLS packet was received.

After spending too much time on this, I think its related to the reverse proxy/load balancer handling the SSL and being incorrectly configured. Help would be much appreciated.

MarkEdmondson1234 commented 2 years ago

Eesh looks rough, not experienced it before and have used it behind load balancer. It looks like a more fundamental curl issue that may be solved by setting options at that level, may be worth searching for that error, sometimes misconfigured servers trip it up, such as not accepting http2 headers. Let me know if anything can be added to help, I guess at googleAuthR or gargle level.

RightChain commented 2 years ago

I'm going to try the solution listed here and hope it doesn't mess with anything.

MarkEdmondson1234 commented 2 years ago

Can I see the Dockerfile you are using to see the system dependencies?

RightChain commented 2 years ago

Here are the relevant sections:

FROM openanalytics/r-ver:3.6.3

RUN apt-get update && apt-get install -y \
    sudo \
    pandoc \
    pandoc-citeproc \
    libcurl4-openssl-dev \
    libcairo2-dev \
    libxt-dev \
    libssl-dev \
    libssh2-1-dev \
    libssl1.0.0 \
    libxml2-dev \ 
    libpq-dev \
    libmpfr-dev \
    libjq-dev \
    libglpk-dev \
    libjpeg-dev \
    libgdal-dev \
    tzdata \
    && apt-get update && apt-get upgrade -y\
    && apt-get clean && apt-get autoremove --purge
#RUN apt-get update && apt-get install -y default-jdk
#libcurl4-gnutls-dev \

RUN a="install.packages(c('curl','shiny','remotes','jsonlite','RColorBrewer','dplyr','data.table','RPostgres','Rcpp' )); \
install.packages(c('shinyWidgets','shinyjs','shinyBS','shinycssloaders','DT','plotly'));  \ 
remotes::install_github('cloudyr/googleCloudStorageR', upgrade='never'); \
remotes::install_github('Displayr/flipTime', upgrade='never'); \
install.packages(c('gmp','sp','geosphere','leaflet','visNetwork','forecast','lubridate','xts','zoo','ClusterR','gtools','readxl')); \
install.packages(c('CVXR','qap','Rglpk','rhandsontable')); \
install.packages(c('googleway','shinydashboard','lpSolve','shinybusy')); \
install.packages(c('moments')); \
remotes::install_github('ropensci/writexl', upgrade='never');" \
&& R -e "$a"

I originally included libcurl4-gnutls-dev but switched it with libcurl4-openssl-dev as the linked solution suggests. I'm still testing this. I left out where I pull and install my own packages which have a couple more R dependencies.

MarkEdmondson1234 commented 2 years ago

What's the FROM image? I have some images with this package installed upon that may help troubleshoot

RightChain commented 2 years ago

It looks like including libcurl4-openssl-dev in my Docker file has fixed the problem. I've also cleaned up my above comments now that I think its solved. Thank you for your support on this!

MarkEdmondson1234 commented 2 years ago

Great :)