localHub=TRUE not sufficient for offline use

[vjcitn]

manpage comment If working offline, add argument ‘localHub=TRUE’ to work with a local, non-updated hub; doesn't hold for my mac, can't test on other platforms as all are connected

> eh = ExperimentHub(localHub=TRUE)
Error: failed to connect to local data base
  database: ‘/Users/stvjc/Library/Caches/ExperimentHub/176be364dd2d7_experimenthub.sqlite3’
  reason: invalid version specification ‘Bioconductor version cannot be validated; no internet connection?’
In addition: Warning messages:
1: In file(con, "r") :
  URL 'https://bioconductor.org/config.yaml': status was 'Couldn't resolve host name'
2: In file(con, "r") :
  URL 'http://bioconductor.org/config.yaml': status was 'Couldn't resolve host name'
> sessionInfo()
R version 3.6.1 Patched (2019-08-15 r77002)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] ExperimentHub_1.11.6 AnnotationHub_2.17.9 BiocFileCache_1.9.1 
[4] dbplyr_1.4.2         BiocGenerics_0.31.5 

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.2                    pillar_1.4.2                 
 [3] compiler_3.6.1                BiocManager_1.30.4           
 [5] later_0.8.0                   tools_3.6.1                  
 [7] zeallot_0.1.0                 digest_0.6.20                
 [9] bit_1.1-14                    RSQLite_2.1.2                

[lshep]

Thanks for reporting this. I'll look into it

[lshep]

@vjcitn Any chance of providing a traceback from your system too?

[vjcitn]

is this enough?

Error: failed to connect to local data base
  database: ‘/Users/stvjc/Library/Caches/ExperimentHub/176be364dd2d7_experimenthub.sqlite3’
  reason: invalid version specification ‘Bioconductor version cannot be validated; no internet connection?’
In addition: Warning messages:
1: In file(con, "r") :
  URL 'https://bioconductor.org/config.yaml': status was 'Couldn't resolve host name'
2: In file(con, "r") :
  URL 'http://bioconductor.org/config.yaml': status was 'Couldn't resolve host name'

Enter a frame number, or 0 to exit   

1: ExperimentHub(localHub = TRUE)
2: .Hub("ExperimentHub", hub, cache, proxy, localHub, ...)
3: .db_uid0(db_path, db_date)
4: tryCatch({
    uid <- .uid0(path, .date)
    sort(uid)
}, error = function(
5: tryCatchList(expr, classes, parentenv, handlers)
6: tryCatchOne(expr, names, parentenv, handlers[[1]])
7: value[[3]](cond)

Selection: 

[lshep]

that helps yes. thank you

[vjcitn]

failing here

BiocManager:::isDevel() Error: invalid version specification ‘Bioconductor version cannot be validated; no internet connection?’ In addition: Warning messages:

inside AnnotationHub:::.uid0

lookslike isDevel() calls need to be conditioned

[lshep]

@mtmorgan / @LiNk-NY I thought isDevel could work when it is offline?

[mtmorgan]

apparently not ;) but even if it handled this gracefully (e.g., NA and a warning) you'd still be stuck not knowing whether this is devel or not. More robust would be to ask for specific version BiocManager::version(), which I believe always works.

[lshep]

But I still have no way of knowing if that version is release or devel - which was the point of using isDevel

[vjcitn]

It sounds to me as if one has to check value of version() against the various relevant release versions, and if the version in use is not among them, deduce "devel".

On Thu, Sep 19, 2019 at 11:18 AM lshep notifications@github.com wrote:

But I still have no way of knowing if that version is release or devel -

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Bioconductor/ExperimentHub/issues/6?email_source=notifications&email_token=ABDI5QWZ7PUPVZSWM5DO7DTQKOJ37A5CNFSM4IYLJIRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7D26MQ#issuecomment-533180210, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDI5QUYWUU66U67C3C365LQKOJ37ANCNFSM4IYLJIRA .

-- The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance  HelpLine at http://www.partners.org/complianceline http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[lshep]

But is that information available offline?

[vjcitn]

I think the current version of .uid0 could be coded with knowledge of the version that it is supplied with -- that's how you get the possible release values. This may be unprecedented -- the source code would have to change with each release, to accommodate a new non-devel version.

[lshep]

I think it may actually be more of an issue with my conditional before it - I assumed AnnotatinoHubOption and didnt take into account ExpeirmentHub reuses this code

[lshep]

I look into it more -

@vjcitn do you know if the LocalHub option works with AnnotaitonHub for you or is that broken as well?

[vjcitn]

> library(AnnotationHub)
5/38 packages newly attached/loaded, see sessionInfo() for details.
> x = AnnotationHub(localHub=TRUE)
Error: failed to connect to local data base
  database: ‘/Users/stvjc/Library/Caches/AnnotationHub/771f444c34b1_annotationhub.sqlite3’
  reason: invalid version specification ‘Bioconductor version cannot be validated; no internet connection?’
In addition: Warning messages:
1: In file(con, "r") :
  URL 'https://bioconductor.org/config.yaml': status was 'Couldn't resolve host name'
2: In file(con, "r") :
  URL 'http://bioconductor.org/config.yaml': status was 'Couldn't resolve host name'

Enter a frame number, or 0 to exit   

1: AnnotationHub(localHub = TRUE)
2: .Hub("AnnotationHub", hub, cache, proxy, localHub, ...)
3: .db_uid0(db_path, db_date)
4: tryCatch({
    uid <- .uid0(path, .date)
    sort(uid)
}, error = function(
5: tryCatchList(expr, classes, parentenv, handlers)
6: tryCatchOne(expr, names, parentenv, handlers[[1]])
7: value[[3]](cond)

[lshep]

neat... okay thanks - i'll work on this tonight

[vjcitn]

just checking in here ... any progress?

[lshep]

Still working on it - I'm trying to think of a more elegant solution - The tricky part is the reused code base between AnnotationHub/ExperimentHub - i don't necessarily want to hard code a Bioc version into the code that would have to be updated per release and need the information to be available offline - I'm thinking making sure the HubOption is set and passing an argument for which hub is calling the functions might be the best tactic but trying to map out possible consequences of this before implementing

[lshep]

I'm hoping to have this resolved within the next few days tho -

[lshep]

I think I have a fix that I plan to push up tomorrow. Just want to do some additional testing tomorrow morning to be sure.

[lshep]

AnnotationHub 2.17.10 should resolve this issue. It was just pushed up and should propagate tomorrow.

[saeidamiri1]

We have to run ExperimentHub in HTC which runs code offline, the cache is there but it does not read, I appreciate any helps

ExperimentHub(localHub=TRUE) Error: failed to connect to local data base database: ‘/home/sam/.cache/R/ExperimentHub/24dc84dbf615e_experimenthub.sqlite3’ reason: invalid version specification ‘Bioconductor version cannot be validated; no internet connection?’

[lshep]

@saeidamiri1 I'll look into this right away. Could you provide your sessionInfo() and if possible a traceback() after the error occurs. This would speed the debugging process greatly.

[saeidamiri1]

@lshep Thanks for a prompt response.

ExperimentHub(localHub=TRUE) Error: failed to connect to local data base database: ‘/home/samamiri/.cache/R/ExperimentHub/24dc84dbf615e_experimenthub.sqlite3’ reason: invalid version specification ‘Bioconductor version cannot be validated; no internet connection?’ In addition: Warning messages: 1: In file(con, "r") : URL 'https://bioconductor.org/config.yaml': Timeout of 60 seconds was reached 2: In file(con, "r") : URL 'http://bioconductor.org/config.yaml': Timeout of 60 seconds was reached traceback() 8: stop("failed to connect to local data base", "\n database: ", sQuote(path), "\n reason: ", conditionMessage(err), call. = FALSE) 7: value[3L] 6: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 5: tryCatchList(expr, classes, parentenv, handlers) 4: tryCatch({ uid <- .uid0(path, .date, localHub) sort(uid) }, error = function(err) { stop("failed to connect to local data base", "\n database: ", sQuote(path), "\n reason: ", conditionMessage(err), call. = FALSE) }) 3: .db_uid0(db_path, db_date, localHub) 2: .Hub("ExperimentHub", hub, cache, proxy, localHub, ask, ...) 1: ExperimentHub(localHub = TRUE) sessionInfo() R version 4.1.2 (2021-11-01) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Rocky Linux 8.4 (Green Obsidian)

Matrix products: default BLAS/LAPACK: /cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/flexiblas/3.0.4/lib64/libflexiblas.so.3.0

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] ExperimentHub_2.2.1 AnnotationHub_3.2.2 BiocFileCache_2.2.1 [4] dbplyr_2.1.1 BiocGenerics_0.40.0

loaded via a namespace (and not attached): [1] KEGGREST_1.34.0 tidyselect_1.1.2
[3] BiocVersion_3.14.0 purrr_0.3.4
[5] vctrs_0.3.8 generics_0.1.2
[7] htmltools_0.5.2 stats4_4.1.2
[9] yaml_2.3.5 utf8_1.2.2
[11] interactiveDisplayBase_1.32.0 blob_1.2.2
[13] rlang_1.0.2 pillar_1.7.0
[15] later_1.3.0 withr_2.5.0
[17] glue_1.6.2 DBI_1.1.2
[19] rappdirs_0.3.3 bit64_4.0.5
[21] GenomeInfoDbData_1.2.7 lifecycle_1.0.1
[23] zlibbioc_1.40.0 Biostrings_2.62.0
[25] memoise_2.0.1 Biobase_2.54.0
[27] IRanges_2.28.0 fastmap_1.1.0
[29] httpuv_1.6.5 GenomeInfoDb_1.30.1
[31] curl_4.3.2 fansi_1.0.3
[33] AnnotationDbi_1.56.2 Rcpp_1.0.8.3
[35] xtable_1.8-4 promises_1.2.0.1
[37] filelock_1.0.2 BiocManager_1.30.16
[39] cachem_1.0.6 S4Vectors_0.32.4
[41] XVector_0.34.0 mime_0.12
[43] bit_4.0.4 png_0.1-7
[45] digest_0.6.29 dplyr_1.0.8
[47] shiny_1.7.1 cli_3.2.0
[49] tools_4.1.2 bitops_1.0-7
[51] magrittr_2.0.2 RCurl_1.98-1.6
[53] tibble_3.1.6 RSQLite_2.2.11
[55] crayon_1.5.1 pkgconfig_2.0.3
[57] ellipsis_0.3.2 assertthat_0.2.1
[59] httr_1.4.2 R6_2.5.1
[61] compiler_4.1.2

[lshep]

@LiNk-NY -- tagging you here too because it seems to come from the offline use of BiocManager::version . I'm going to do some more testing/digging/debugging but as a heads up

[LiNk-NY]

Hi @saeidamiri1 Please see the Offline Use section in the BiocManager vignette. Let us know if that helps. Best regards, Marcel

[lshep]

I'll look at adding this as a reference section to the hubs as well then; or some better work around in the hub code.

[lshep]

I've added sections in the man page and a message when localHub=TRUE to point to the BiocManager section of the vignette for offline use