Connecting to Ensembl: Internal Server Error (HTTP 500).

[JonatanFernandez]

Hello!

Lately I've having issues when trying to connect to Ensembl. I did check the webpage and it's working.... My code used to work, but lately I've been having issues with both the bioconductor and the github version of the package.

require(biomaRt)
listMarts(host="www.ensembl.org")

Error in bmRequest(request = request, ssl.verifypeer = ssl.verifypeer, : Internal Server Error (HTTP 500).

Moreover, if I try to execute

useMart(biomart="ensembl")

Sometimes happens this:

Error in bmRequest(request = request, ssl.verifypeer = ssl.verifypeer, : Internal Server Error (HTTP 500).

And sometimes it get stalled and if I try to terminate the execution it never ends.

sessionInfo() R version 3.6.1 (2019-07-05) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 19.04

Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.8.0

locale: [1] LC_CTYPE=en_IL.UTF-8 LC_NUMERIC=C LC_TIME=en_IL.UTF-8 LC_COLLATE=en_IL.UTF-8 LC_MONETARY=en_IL.UTF-8 LC_MESSAGES=en_IL.UTF-8 LC_PAPER=en_IL.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_IL.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] parallel stats4 stats graphics grDevices utils datasets methods base

other attached packages: [1] biomaRt_2.41.7 IHW_1.12.0 DESeq2_1.24.0 SummarizedExperiment_1.14.0 DelayedArray_0.10.0 BiocParallel_1.18.0 matrixStats_0.54.0 Biobase_2.44.0
[9] GenomicRanges_1.36.0 GenomeInfoDb_1.20.0 IRanges_2.18.1 S4Vectors_0.22.0 BiocGenerics_0.30.0 forcats_0.4.0

loaded via a namespace (and not attached): [1] fs_1.3.1 bitops_1.0-6 usethis_1.5.1 devtools_2.1.0 bit64_0.9-7 RColorBrewer_1.1-2 progress_1.2.2 httr_1.4.0 rprojroot_1.3-2
[10] tools_3.6.1 backports_1.1.4 R6_2.4.0 rpart_4.1-13 Hmisc_4.2-0 DBI_1.0.0 lazyeval_0.2.2 colorspace_1.4-1 nnet_7.3-12
[19] withr_2.1.2 tidyselect_0.2.5 gridExtra_2.3 prettyunits_1.0.2 processx_3.4.0 bit_1.1-14 curl_3.3 compiler_3.6.1 fdrtool_1.2.15
[28] cli_1.1.0 htmlTable_1.13.1 desc_1.2.0 slam_0.1-45 scales_1.0.0 checkmate_1.9.4 genefilter_1.66.0 callr_3.3.0 stringr_1.4.0
[37] digest_0.6.20 foreign_0.8-71 XVector_0.24.0 base64enc_0.1-3 pkgconfig_2.0.2 htmltools_0.3.6 sessioninfo_1.1.1 lpsymphony_1.12.0 htmlwidgets_1.3
[46] rlang_0.4.0 rstudioapi_0.10 RSQLite_2.1.1 acepack_1.4.1 dplyr_0.8.3 RCurl_1.95-4.12 magrittr_1.5 GenomeInfoDbData_1.2.1 Formula_1.2-3
[55] Matrix_1.2-17 Rcpp_1.0.1 munsell_0.5.0 stringi_1.4.3 zlibbioc_1.30.0 pkgbuild_1.0.3 grid_3.6.1 blob_1.2.0 crayon_1.3.4
[64] lattice_0.20-38 splines_3.6.1 annotate_1.62.0 hms_0.5.0 locfit_1.5-9.1 zeallot_0.1.0 knitr_1.23 ps_1.3.0 pillar_1.4.2
[73] geneplotter_1.62.0 pkgload_1.0.2 XML_3.98-1.20 glue_1.3.1 latticeExtra_0.6-28 remotes_2.1.0 data.table_1.12.2 vctrs_0.2.0 testthat_2.1.1
[82] gtable_0.3.0 purrr_0.3.2 assertthat_0.2.1 ggplot2_3.2.0 xfun_0.8 xtable_1.8-4 survival_2.44-1.1 tibble_2.1.3 AnnotationDbi_1.46.0
[91] memoise_1.1.0 cluster_2.1.0

Thanks a lot for the great job!

[borauyar]

I am getting the same error recently with the following command: biomaRt::useMart("ensembl", dataset = "hsapiens_gene_ensembl") The error is thrown somewhat randomly, but quite often.

[grimbough]

This issue is related to server load at Ensembl, and seems to be increasing in frequency since release 96. I have found that using the mirror sites is often beneficial as these seem to be under less load.

I have also added a caching mechanism to the developmental version biomaRt. If a query times out with the 500 error it should store the results it's obtained upto that point in a tempoary directory. It doesn't solve the timeout problem, but if you rerun the query it will load them from disk and keep going from that point.

This is pretty experimental, so if you're using the code from Github can you let me know if this seems to be working.