Server certificate for 'https://hedgehog.fhcrc.org:443'

[jennybc]

I can't pass the devtools tests locally because this install_bioc() command fails.

  # This package has no dependencies or compiled code and is old
  install_bioc("MeasurementError.cor", quiet = TRUE)

  expect_silent(packageDescription("MeasurementError.cor"))
  expect_equal(packageDescription("MeasurementError.cor")$RemoteType, "bioc")

Run interactively, here's what I get:

> install_bioc("MeasurementError.cor", quiet = TRUE)
Error validating server certificate for 'https://hedgehog.fhcrc.org:443':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: *.fhcrc.org
 - Valid: from Sat, 18 Oct 2014 16:52:00 GMT until Fri, 23 Nov 2018 15:15:23 GMT
 - Issuer: http://certs.godaddy.com/repository/, GoDaddy.com, Inc., Scottsdale, Arizona, US
 - Fingerprint: 55:64:9d:ce:78:a9:b0:b4:2f:df:4c:c6:59:b1:51:56:bb:5c:44:6a
(R)eject, accept (t)emporarily or accept (p)ermanently? 

Even when I accept, the R process hangs and I have to force quit RStudio entirely. I've tried other mirrors with the same result.

When I try to accept the fingerprint interactively in the shell, as suggested in #1307:

svn info https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/madman/Rpacks

I am challenged for a password and therefore fail to close the deal:

jenny@2015-mbp tmp $ svn info https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/madman/Rpacks
Error validating server certificate for 'https://hedgehog.fhcrc.org:443':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: *.fhcrc.org
 - Valid: from Sat, 18 Oct 2014 16:52:00 GMT until Fri, 23 Nov 2018 15:15:23 GMT
 - Issuer: http://certs.godaddy.com/repository/, GoDaddy.com, Inc., Scottsdale, Arizona, US
 - Fingerprint: 55:64:9d:ce:78:a9:b0:b4:2f:df:4c:c6:59:b1:51:56:bb:5c:44:6a
(R)eject, accept (t)emporarily or accept (p)ermanently? p
Authentication realm: <https://hedgehog.fhcrc.org:443> The bioconductor Subversion Repository
Password for 'jenny': 
svn: E175002: Unable to connect to a repository at URL 'https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/madman/Rpacks'
svn: E175002: Server sent unexpected return value (500 Internal Server Error) in response to OPTIONS request for 'https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/madman/Rpacks'

Seems related to this thread on BioConductor which is also unresolved:

https://support.bioconductor.org/p/86107/

@jimhester

[jimhester]

svn info --username readonly --password readonly https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/madman/Rpacks

should allow you to accept the certificate.

@mtmorgan Do you know of anything we can do to remedy this? I get the prompt (and error) just using the command line tools on a fresh install (or by moving ~/.subversion), so this is not just an R or devtools issue.

[jennybc]

Thanks @jimhester that DOES allow me to accept the certificate. Well, not really "accept". There was nothing interactive about it, but it seems to have accomplished what was needed.

But the installation of MeasurementError.cor still fails, like so:

> install_bioc("MeasurementError.cor", quiet = FALSE)
Downloading Bioconductor repo MeasurementError.cor
/usr/bin/svn co --username readonly --password readonly https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/MeasurementError.cor /var/folders/vt/4sdxy0rd1b3b65nqssx4sx_h0000gn/T//RtmpYiFiBW/file434665cdf239
Installing MeasurementError.cor
'/Library/Frameworks/R.framework/Resources/bin/R' CMD INSTALL '/private/var/folders/vt/4sdxy0rd1b3b65nqssx4sx_h0000gn/T/RtmpYiFiBW/file434665cdf239' 
Installation failed: argument is of length zero

[jimhester]

I think the SVN certificate it is still messed up on your machine actually, can you try the following in a shell.

mv ~/.subversion{,.bak}
svn info --username readonly --password readonly https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/madman/Rpacks
# accept it permanetly
/usr/bin/svn co --username readonly --password readonly https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/MeasurementError.cor /tmp/test

Then verify that /tmp/test contains the checked out version of that package. If that works successfully than the test should also work.

[jennybc]

It is true that the package was not being downloaded. So I did the svn / certificate stuff above. All seemed to be successful and yes there was interactive acceptance this time. Yes /tmp/test contains the checked out version of MeasurementError.cor.

However I was getting the exact same failure with install_bioc("MeasurementError.cor", quiet = FALSE), interactively and with Check in RStudio build pane.

I note that if I change the library paths in this test, I can pass tests now:

## before
.libPaths(lib)
## after
.libPaths(c(lib, libpath))

[jimhester]

Hmm, works ok for me, does making this change fix it for you?

diff --git a/tests/testthat/test-bioconductor.r b/tests/testthat/test-bioconductor.r
index c497e08..862c5fb 100644
--- a/tests/testthat/test-bioconductor.r
+++ b/tests/testthat/test-bioconductor.r
@@ -38,6 +38,6 @@ test_that("install_bioc", {
   # This package has no dependencies or compiled code and is old
   install_bioc("MeasurementError.cor", quiet = TRUE)

-  expect_silent(packageDescription("MeasurementError.cor"))
-  expect_equal(packageDescription("MeasurementError.cor")$RemoteType, "bioc")
+  expect_silent(desc <- packageDescription("MeasurementError.cor", lib.loc = .libPaths()))
+  expect_equal(desc$RemoteType, "bioc")
 })

[jennybc]

No, that does not fix it. Possibly relevant: I don't allow anything but the base and Recommended packages to live in my default library. Is it possible you have more there?

[jimhester]

No I use a personal library as well ~/Library/R/3.3/library. libPaths() is just to the temporary directory is so that the bioconductor installation is completely isolated, so using the old libpaths as well seems wrong. Do you still get the same results using R --vanilla -e 'devtools::test()'?

[jennybc]

Then I get this, since devtools itself cannot be found:

jenny@2015-mbp devtools $ R --vanilla -e 'devtools::test()'

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> devtools::test()
Error in loadNamespace(name) : there is no package called ‘devtools’
Calls: :: ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>
Execution halted

I think we are on to something w/r/t me because I have other problems with devtools tests depending on how I run them and they are all plausibly related to library paths.

[jennybc]

For example, here what I get with devtools::test() in the R console (with the modified lib path in the BioC test):

> test()
Loading devtools
Testing devtools
bioc: ..............
Check: ....
getrootdir: ..
git: ....
git usage and GitHub connections: ................................
GitHub: ........................
Infrastructure: ...................
Install specific version: ......
Install: ..
remote-metadata: .....
remote_deps: .......................12.3
sort: ...
Test: ..
Uninstall: ..4
update.package_deps: ...
Vignettes: .......................
...

Failed ----------------------------------------------------------------------
1. Failure: package2remotes looks for the DESCRIPTION in .libPaths (@test-remotes.r#79) 
package2remote("testTest")$sha not equal to NA.
1/1 mismatches
x[1]: "0.1"
y[1]: NA

2. Failure: package2remotes looks for the DESCRIPTION in .libPaths (@test-remotes.r#80) 
package2remote("testTest")$sha not equal to NA.
1/1 mismatches
x[1]: "0.1"
y[1]: NA

3. Failure: package2remotes looks for the DESCRIPTION in .libPaths (@test-remotes.r#89) 
package2remote("testTest")$sha not equal to NA.
1/1 mismatches
x[1]: "0.1"
y[1]: NA

4. Error: uninstall() unloads and removes from library (@test-uninstall.r#20) 
there is no package called ‘testHelp’
1: uninstall("testHelp", quiet = TRUE) at /Users/jenny/rrr/devtools/tests/testthat/test-uninstall.r:20
2: remove.packages(pkg$package) at /Users/jenny/rrr/devtools/R/uninstall.r:27
3: find.package(pkgs, lib)
4: stop(gettextf("there is no package called %s", sQuote(pkg)), domain = NA)

DONE ========================================================================

The same tests fail via R CMD check and Rscript -e 'devtools::test("devtools")', but pass cleanly via Rscript -e 'devtools::check("devtools")'.

I can take this to a different issue or slack if you want? As I think we have resolved the Bioconductor issue (sort of).

[mtmorgan]

Hi @jimhester I (continue) to think that the certificate is valid but that the CA certificate bundle @jennybc (and other Mac users?) have does not recognize the authority. I'm told that a slightly less heavy-handed solution is to delete ~/.subversion/auth/svn.ssl.server and then manually accept (permanently) as above svn list --username readonly --password readonly https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/madman/Rpacks maybe substituting --username with a valid svn username.

[jimhester]

@mtmorgan After looking into this a little more you are correct, the SVN client bundled with MacOS doesn't include any CA bundles. MacOS does come with CA bundles, but they are part of the keychain. http://superuser.com/questions/903263/what-does-subversion-use-for-its-ca-list has some options for using the bundled command line SVN.

Alternatively if one is using SVN from homebrew it is linked to openssl. Openssl is also doesn't have the appropriate CA bundle by default, but you can export it from the keychain (as a .pem)

Add it to /usr/local/etc/openssl/certs and run /usr/local/opt/openssl/bin/c_rehash.

This will then prompt you to use the certificate bundle when trying to connect to hedgehog and authenticate it appropriately.

Which is a long winded way of saying this is really a shortcoming of the svn clients for OS X and the CA bundle setup, not a bioconductor issue and the easiest option for users on OS X is to do as suggested, run a svn command against the bioconductor servers and permanently accept the certificate.

[jennybc]

So is a proper svn setup absolutely necessary to test the BioC functionality in devtools?

[mtmorgan]

Two comments, but more from the outsider perspective as I don't know what @jennybc or devtools are trying to accomplish. Bioc packages are distributed in CRAN-style repositories, so svn is only required for source code access. The appropriate repositories for the version of R in use is given by

source("https://biocondcutor.org/biocLite.R")
BiocInstaller::biocinstallRepos()

One could then use this as an argument to install.packages() / devtools::install() or ignore the explicit need to specify repository with biocLite() (which installs Bioconductor, CRAN [via install.packages()] and github [via delegation to devtools] packages). The title for install_bioc() ("Install a package from a Bioconductor repository") is ambiguous, since there are CRAN-style 'repositories' (used in install_cran()) and source control repositories (like github or svn); Bioconductor hosts both. Also, Bioconductor has (and correct functionality depends on) a particular versioning system, and it is inappropriate for instance to install the current svn 'trunk' packages on the current release version of R. BiocInstaller and our repositories enforce this system, but devtools::install_github() will lead to invalid installations.

Maybe slightly less satisfactory is github.com/Bioconductor-mirror and branches @release-3.4, etc. which can be accessed as, e.g., devtools::install_github("Bioconductor-mirror/"). Unsatisfactory because parts of the mirror are sometimes out of sync with svn, and because svn is really the cannonical source code repository for Bioconductor packages.

See also https://github.com/hadley/devtools/issues/1240

[lock[bot]]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/