search

Bioconductor / GenomicDataCommons

Provide R access to the NCI Genomic Data Commons portal.
http://bioconductor.github.io/GenomicDataCommons/
83 stars 23 forks source link

guidance on solving SSL error? #35

Closed vjcitn closed 7 years ago

vjcitn commented 7 years ago

source 0.99.7 %vjcair> R CMD build GenomicDataCommons

seandavi commented 7 years ago

Does this work for you, Vince?

httr::GET('https://gdc-api.nci.nih.gov/status')

Output should look something like:

....
{
  "commit": "9f568bab7dd61f17e24e8c9fa6e62235233d369e",
  "status": "OK",
  "tag": "1.5.0",
  "version": 1
seandavi commented 7 years ago

Hi, @vjcitn. This looks like a problem with ssl verification. After starting R and loading httr, can you do this and then install the package in the same R session?

httr::set_config(httr::config(ssl_verifypeer=0L))

This has been seen elsewhere including https://github.com/swirldev/swirl/issues/475. I think the ultimate fix is to update the SSL package for your system. What OS are you on?

seandavi commented 7 years ago

And another reference with suggestions for linux. http://stackoverflow.com/questions/35055715/ssl-connect-error-in-httr-curl

vjcitn commented 7 years ago

Thanks for looking into this. I am on Mac OSX 10.11.6 .

httr::GET('https://gdc-api.nci.nih.gov/status') Error in curl::curl_fetch_memory(url, handle = handle) : SSL connect error

Enter a frame number, or 0 to exit

1: httr::GET("https://gdc-api.nci.nih.gov/status") 2: request_perform(req, hu$handle$handle) 3: request_fetch(req$output, req$url, handle) 4: request_fetch.write_memory(req$output, req$url, handle) 5: curl::curl_fetch_memory(url, handle = handle)

Selection: 0

sessionInfo() R Under development (unstable) (2017-03-02 r72298) Platform: x86_64-apple-darwin13.4.0 (64-bit) Running under: OS X El Capitan 10.11.6

Matrix products: default BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] httr_1.2.1 curl_2.3 rmarkdown_1.3

loaded via a namespace (and not attached): [1] compiler_3.4.0 backports_1.0.5 R6_2.2.0 magrittr_1.5
[5] rprojroot_1.2 tools_3.4.0 htmltools_0.3.5 Rcpp_0.12.9
[9] stringi_1.1.2 knitr_1.15.1 stringr_1.1.0 digest_0.6.12
[13] evaluate_0.10

seandavi commented 7 years ago

Thanks, @vjcitn. If you do this before the httr::GET, do you still get the error?

httr::set_config(httr::config(ssl_verifypeer=0L))
vjcitn commented 7 years ago

indeed the error occurs after set_config

httr::set_config(httr::config(ssl_verifypeer=0L))

httr::GET('https://gdc-api.nci.nih.gov/status')

Error in curl::curl_fetch_memory(url, handle = handle) :

Enter a frame number, or 0 to exit

1: httr::GET("https://gdc-api.nci.nih.gov/status")

2: request_perform(req, hu$handle$handle)

3: request_fetch(req$output, req$url, handle)

4: request_fetch.write_memory(req$output, req$url, handle)

5: curl::curl_fetch_memory(url, handle = handle)

On Fri, Mar 3, 2017 at 4:30 PM, Sean Davis notifications@github.com wrote:

Thanks, @vjcitn https://github.com/vjcitn. If you do this before the httr::GET, do you still get the error?

httr::set_config(httr::config(ssl_verifypeer=0L))

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Bioconductor/GenomicDataCommons/issues/35#issuecomment-284075338, or mute the thread https://github.com/notifications/unsubscribe-auth/AEaOwiDbzIudQzSGM4VG1SsgsuZOX_cNks5riIZwgaJpZM4MRUDW .

seandavi commented 7 years ago

Sorry for the runaround. I have not been able to reproduce this locally, but two of you are seeing the same issue, so I know we need to track it down.

And how about:

httr::set_config(httr::config(ssl_verifypeer=0L, ssl_verifyhost=0L))

If that doesn't do the trick, how about:

httr::set_config(httr::config(ssl_verifypeer=0L, ssl_verifyhost=0L, sslversion=3))

Thanks for the help and patience.

seandavi commented 7 years ago

So, it looks like an update to openssl is necessary. See 34b95b09b for details; a vignette section has attempted to outline the problem and the potential solutions. On mac os, it appears that using brew to install openssl, link it, and then reinstall curl and httr should do the trick.

Let me know how it goes.

vjcitn commented 7 years ago

I hate to drag this out so long. I upgraded openssl and attempted to inform the package build environment about where to look for the current openssl

%vjcair> openssl version -a

OpenSSL 1.0.2k 26 Jan 2017

built on: reproducible build, date unspecified

platform: darwin64-x86_64-cc

options: bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) idea(int) blowfish(idx)

compiler: clang -I. -I.. -I../include -fPIC -fno-common -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM

OPENSSLDIR: "/usr/local/etc/openssl"

%vjcair>

%vjcair> pwd

/Users/stvjc

%vjcair> vi SSLSTUFF

%vjcair> export LDFLAGS=-L/usr/local/opt/openssl/lib:$LDFLAGS

%vjcair> export CPPFLAGS=-I/usr/local/opt/openssl/include:$CPPFLAGS

%vjcair> export PKG_CONFIG_PATH=/usr/local/opt/openssl/lib/pkgconfig:$PKG_CONFIG_PATH

then using R CMD INSTALL --clean --preclean for current downloads of curl and httr packages, we still have

httr::set_config(httr::config(ssl_verifypeer=0L, ssl_verifyhost=0L, sslversion=3))

httr::GET('https://gdc-api.nci.nih.gov/status')

Error in curl::curl_fetch_memory(url, handle = handle) :

Enter a frame number, or 0 to exit

1: httr::GET("https://gdc-api.nci.nih.gov/status")

2: request_perform(req, hu$handle$handle)

3: request_fetch(req$output, req$url, handle)

4: request_fetch.write_memory(req$output, req$url, handle)

5: curl::curl_fetch_memory(url, handle = handle)

Does R itself need to be rebuilt with the newer openSSL?

On Sat, Mar 4, 2017 at 4:51 PM, Sean Davis notifications@github.com wrote:

So, it looks like an update to openssl is necessary. See 34b95b0 https://github.com/Bioconductor/GenomicDataCommons/commit/34b95b09b3a60488792c63e26caec96b9f8e535c for details; a vignette section has attempted to outline the problem and the potential solutions. On mac os, it appears that using brew to install openssl, link it, and then reinstall curl and httr should do the trick.

Let me know how it goes.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Bioconductor/GenomicDataCommons/issues/35#issuecomment-284185701, or mute the thread https://github.com/notifications/unsubscribe-auth/AEaOwiaYJlMfBxMjVwkOidBoshrN-Nesks5ridzXgaJpZM4MRUDW .

seandavi commented 7 years ago

Thanks, @vjcitn. I really appreciate your patience in working through this.

@hadley, @jeroenooms, or @kbroman, any thoughts on this issue? Any insight would be greatly appreciated.

vjcitn commented 7 years ago

Here is a solution. It is likely a good idea to verify that command-line curl gets a good response

curl -v -I https://gdc-api.nci.nih.gov/status

the response is relatively long but begins with

`%vjcair> curl -v -I https://gdc-api.nci.nih.gov/status

If that is not working it does not seem that the R interface will work either.

In order to get to this point, I needed to get a relatively current instance of openssl and install curl 7.53.1. The openssl was handled with brew upgrade openssl, and while configuring curl I needed to have /usr/local/opt/openssl/lib/pkgconfig in PKG_CONFIG_PATH. Once curl is built and installed in a personal folder (value of $MYCURLDIR, say), it seems to be necessary to have $MYCURLDIR/lib/ in DYLD_LIBRARY_PATH. (But maybe not.) With PATH properly set to find this new curl and curl-config, we have

%vjcair> curl --version curl 7.53.1 (x86_64-apple-darwin16.4.0) libcurl/7.53.1 OpenSSL/1.0.2k zlib/1.2.8 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy

At this point I can install the R package for curl, but, in my case, not without putting

-L/Users/stvjc/ExternalSoft/CURL-7.53-DIST/lib -lcurl

as a component of LDFLAGS in $HOME/.R/Makevars

Once curl package is installed with a suitable openssl and libcurl, we can do

> httr::GET('https://gdc-api.nci.nih.gov/status') Response [https://gdc-api.nci.nih.gov/status] Date: 2017-03-05 14:59 Status: 200 Content-Type: application/json Size: 110 B { "commit": "9f568bab7dd61f17e24e8c9fa6e62235233d369e", "status": "OK", "tag": "1.5.0", "version": 1

jeroen commented 7 years ago

Is this resolved? I have not been able to reproduce this error on MacOS 10.12 or MacOS 10.9 using any version of libcurl or openssl...

vjcitn commented 7 years ago

I consider it resolved.

seandavi commented 7 years ago

At a very high level, upgrade of openssl, upgrade/reinstallation of libcurl, and then re-installation in R of httr and curl seems to have been the fix.

jeroen commented 7 years ago

I don't think that was the problem. Note that OSX native libcurl does not use openssl, but rather apple's native crypto implementation SecureTransport.

It could very well have been a temporary server misconfiguration.

seandavi commented 7 years ago

Thanks, @jeroenooms. In any case, it appears that Vince and Martin were able to come to a solution for Vince.