search

gladkia / igvR

An R Bioconductor package providing interactive connections to igv.js (the Integrative Genomics Viewer) in a web browser
MIT License
42 stars 4 forks source link

`host` parameter ignored by igvR constructor #17

Closed genec1 closed 2 years ago

genec1 commented 2 years ago

The host parameter specified by the igvR constructor appears to be ignored and this is leading to errors displaying tracks:

igv <- igvR(host = "192.168.0.1", quiet=TRUE)
igv@uri
# "http://localhost:15002"

# ... setup genome and tracks
displayTrack(igv, my_bam_track)
# bam url: http://localhost:15003?/tmp/Rtmp0ZcCIP/filec49620e6c76a.bam 
# bam track height: 50

Note that the uri slot and the displayTrack message both say "localhost" instead of the host value I specified. The actual viewer also displays an error: Error accessing resource: http://localhost:15003?/tmp/Rtmp0ZcCIP/filec49620e6c76a.bam.bai Status: 0

sessionInfo() is:


R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.5 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8        LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8   
 [6] LC_MESSAGES=C.UTF-8    LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C           LC_TELEPHONE=C        
[11] LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] BSgenome.Hsapiens.UCSC.hg38_1.4.3 BSgenome_1.58.0                   rtracklayer_1.50.0               
 [4] igvR_1.15.3                       BrowserViz_2.12.0                 httpuv_1.6.5                     
 [7] jsonlite_1.7.3                    GenomicAlignments_1.26.0          SummarizedExperiment_1.20.0      
[10] Biobase_2.50.0                    MatrixGenerics_1.2.1              matrixStats_0.61.0               
[13] reticulate_1.16                   data.table_1.14.2                 forcats_0.5.0                    
[16] stringr_1.4.0                     dplyr_1.0.7                       purrr_0.3.4                      
[19] readr_1.3.1                       tidyr_1.1.1                       tibble_3.1.6                     
[22] ggplot2_3.3.2                     tidyverse_1.3.0                   Rsamtools_2.6.0                  
[25] Biostrings_2.58.0                 XVector_0.30.0                    GenomicRanges_1.42.0             
[28] GenomeInfoDb_1.26.7               IRanges_2.24.1                    S4Vectors_0.28.1                 
[31] BiocGenerics_0.36.1              

loaded via a namespace (and not attached):
 [1] colorspace_2.0-2         ellipsis_0.3.2           fs_1.5.2                 rstudioapi_0.13          farver_2.1.0            
 [6] bit64_4.0.5              AnnotationDbi_1.52.0     fansi_1.0.2              lubridate_1.7.9          xml2_1.3.3              
[11] cachem_1.0.6             knitr_1.29               splitstackshape_1.4.8    seqLogo_1.56.0           broom_0.7.0             
[16] dbplyr_2.1.1             compiler_4.0.2           httr_1.4.2               backports_1.4.1          assertthat_0.2.1        
[21] Matrix_1.4-0             fastmap_1.1.0            cli_3.1.1                later_1.3.0              prettyunits_1.1.1       
[26] tools_4.0.2              gtable_0.3.0             glue_1.6.1               GenomeInfoDbData_1.2.4   rappdirs_0.3.3          
[31] Rcpp_1.0.8               cellranger_1.1.0         vctrs_0.3.8              xfun_0.16                rvest_0.3.6             
[36] lifecycle_1.0.1          XML_3.99-0.8             zlibbioc_1.36.0          scales_1.1.1             VariantAnnotation_1.36.0
[41] hms_1.1.1                promises_1.2.0.1         RColorBrewer_1.1-2       curl_4.3.2               memoise_2.0.1           
[46] biomaRt_2.46.3           stringi_1.7.6            RSQLite_2.2.9            GenomicFeatures_1.42.3   BiocParallel_1.24.1     
[51] rlang_1.0.0              pkgconfig_2.0.3          bitops_1.0-7             lattice_0.20-45          labeling_0.4.2          
[56] bit_4.0.4                tidyselect_1.1.1         magrittr_2.0.2           R6_2.5.1                 generics_0.1.1          
[61] DelayedArray_0.16.3      DBI_1.1.2                pillar_1.6.5             haven_2.3.1              withr_2.4.3             
[66] RCurl_1.98-1.5           MotifDb_1.32.0           modelr_0.1.8             crayon_1.4.2             utf8_1.2.2              
[71] BiocFileCache_1.14.0     progress_1.2.2           grid_4.0.2               readxl_1.3.1             blob_1.2.2              
[76] reprex_0.3.0             digest_0.6.29            openssl_1.4.6            munsell_0.5.0            viridisLite_0.3.0       
[81] askpass_1.1

EDIT: I manually edited the contents of the uri slot, but am still getting the viewer error. The path in the URL "/tmp/Rtmp0ZcCIP/filec49620e6c76a.bam" is the correct file. When I use lsof on the command line, I notice that rsession is still listening on localhost instead of the ip, so I think that is still the issue.

paul-shannon commented 2 years ago

@genec1 - looking at this now, thanks for the report.

paul-shannon commented 2 years ago

@genec1, The constructor's host argument is a bad idea - my bad idea, left over from a long time ago. The man page on the constructor illuminates this, partly: 

 @param host In practice, this is always "localhost"

The justification: igvR displays in your browser, and it is hard for me to imagine a case when "your browser" runs on anything but localhost. Do you have a use-case that suggests otherwise? I'd be glad to see that, and to see if I can support it.

Even with this historical relic cleaned up, I have not helped you with your problem. Let me make sure I understand what that is. The best way would be for you to include the code with which you construct your GenomicAlignmentsTrack object. Here are two examples from inst/unitTests/test_igvR.R.

First, display an alignment already read into memory:

   little.region <- GRanges(seqnames="21", ranges=IRanges(10400126, 10400326))
   param <- ScanBamParam(which=little.region, what=scanBamWhat())
   x <- readGAlignments(bamFile, use.names=TRUE, param=param)
   track <- GenomicAlignmentTrack("bam demo", x, visibilityWindow=2000000, trackHeight=500)  # 30000 default
   displayTrack(igv, track)

Second, display an alignment hosted elsewhere:

   url <- "https://1000genomes.s3.amazonaws.com/phase3/data/HG02450/alignment/HG02450.mapped.ILLUMINA.bwa.ACB.low_coverage.20120522.bam"
   index <- "https://1000genomes.s3.amazonaws.com/phase3/data/HG02450/alignment/HG02450.mapped.ILLUMINA.bwa.ACB.low_coverage.20120522.bam.bai"
   track <- RemoteAlignmentTrack("remote bam", url, index, visibilityWindow=100000)
   displayTrack(igv, track)

How do you create your track object?

genec1 commented 2 years ago

@genec1, The constructor's host argument is a bad idea - my bad idea, left over from a long time ago. The man page on the constructor illuminates this, partly:

 @param host In practice, this is always "localhost"

The justification: igvR displays in your browser, and it is hard for me to imagine a case when "your browser" runs on anything but localhost. Do you have a use-case that suggests otherwise? I'd be glad to see that, and to see if I can support it.

My particular use case is I’m running the code on an instance on AWS, as my local machine can’t handle the amount of data I’m working with.

Even with this historical relic cleaned up, I have not helped you with your problem. Let me make sure I understand what that is. The best way would be for you to include the code with which you construct your GenomicAlignmentsTrack object. Here are two examples from inst/unitTests/test_igvR.R.

How do you create your track object?

I’m working with local bam files read into GenomicAlignment objects very similar to your first example. I will share actual code when I’m back on my workstation.

Thanks

genec1 commented 2 years ago

Bam track display code:

# libraries
library(igvR)
library(BSgenome.Hsapiens.UCSC.hg38)

# setup igv
igv <- igvR(quiet=TRUE)
setGenome(igv, "hg38")
showGenomicRegion(igv, "chr17:7,663,654-7,692,258")
loc <- getGenomicRegion(igv)
logGR <- with(loc, GRanges(seqnames=chrom, ranges = IRanges(start, end)))
param <- ScanBamParam(which=logGR, what = scanBamWhat())

# setup bam
# <define BAM_FILE_PATH and BAM_TRACK_NAME>
galign <- readGAlignments(BAM_FILE_PATH, use.names=TRUE, param=param)
track <- GenomicAlignmentTrack(BAM_TRACK_NAME, galign)
displayTrack(igv, track)
# bam url: http://localhost:15001?/tmp/Rtmpqy9pbo/fileae83dff0a45.bam
# bam track height: 50

# viewer message: Error accessing resource: http://localhost:15001?/tmp/Rtmpqy9pbo/fileae83dff0a45.bam.bai Status: 0
paul-shannon commented 2 years ago

Thanks @genec1. Could you also give me the exact BAM_FILE_PATH so that, with proper substitutions, I can emulate it here?

On Jan 31, 2022, at 9:03 AM, genec1 @.***> wrote:

Bam track display code:

libraries

library(igvR) library(BSgenome.Hsapiens.UCSC.hg38)

setup igv

igv <- igvR(quiet=TRUE) setGenome(igv, "hg38") showGenomicRegion(igv, "chr17:7,663,654-7,692,258") loc <- getGenomicRegion(igv) logGR <- with(loc, GRanges(seqnames=chrom, ranges = IRanges(start, end))) param <- ScanBamParam(which=logGR, what = scanBamWhat())

setup bam

galign <- readGAlignments(BAM_FILE_PATH, use.names=TRUE, param=param) track <- GenomicAlignmentTrack(BAM_TRACK_NAME, galign) displayTrack(igv, track)

bam url: http://localhost:15001?/tmp/Rtmpqy9pbo/fileae83dff0a45.bam

bam track height: 50

viewer message: Error accessing resource: http://localhost:15001?/tmp/Rtmpqy9pbo/fileae83dff0a45.bam.bai Status: 0

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.

genec1 commented 2 years ago

BAM_FILE_PATH is the full path to a local file on the AWS instance where I'm running R, so I don't think it will be of help to you.

paul-shannon commented 2 years ago

@genec1 - you are right, of course. I will need to name a file on my local system since yours will be invisible to me.

Nevertheless, and though you may think I am being silly, please provide me with the exact full file path you use.

On Jan 31, 2022, at 11:29 AM, genec1 @.***> wrote:

BAM_FILE_PATH is the full path to a local file on the AWS instance where I'm running R, so I don't think it will be of help to you.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.

genec1 commented 2 years ago

No problem. I'm working with multiple files, but here is one: /home/rstudio/s3/HEK293T/SRR844_1.filtered.aligned.unique.bam

paul-shannon commented 2 years ago

Hi @genec1. I am slow on the uptake here. Sorry. Let me ask a few more questions.

1) You built igvR on an aws instance. 2) You start igvR from an R session on that instance. 3a) Do you ask igvR to open a websocket connection, from aws, back to a browser running on your workstation, which is local - that is, you sit in front of it 3b) Or do you start up a web browser which runs on the aws instance, which displays (via X windows, for instance) which displays back to your workstation?

You explained that you use aws because the of the large amount of data that you are working with.

When I finally understand your setup, my response is likely to be this:

o igvR is really meant to be run only out of an R session on your local machine o when I have heavy processing to do, resulting in large genomic data files I want to explore, here is what I do: 1) set up a simple byte-range/CORS webserver on the compute server (your aws instance) 2) create and write your big data, indexed bam, vcf, cram etc files there 3) visualize them through the local igvR

It take a little effort to set up the byte-range/CORS webserver, but only a little. I could help with that.

On Jan 31, 2022, at 11:39 AM, genec1 @.***> wrote:

No problem. I'm working with multiple files, but here is one: /home/rstudio/s3/HEK293T/SRR844_1.filtered.aligned.unique.bam

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.

genec1 commented 2 years ago

1 + 2 + 3a

Specifically, I am coding in a local web browser running RStudio, backed by rstudio-server on the AWS instance. When I initiate igvR in my R session, a new window opens in my local web browser showing IGV. This new window correctly shows the reference genome, but cannot show the programmatically loaded bam data since that is not on localhost relative to my browser session.

As a stopgap, I have been region-filtering the bams on AWS, sftp'ing them to my local machine, and opening them in a local IGV app. This is cumbersome for multiple bams with multiple regions. If you can point me to the byte-range/CORS server info, that would be appreciated!

paul-shannon commented 2 years ago

@genec1 I have a presentation to give tomorrow, so today I can only give you quick & dirty instructions on creating a python flask webserver for serving up indexed genomic files, bam and vcf, for instance. I will polish these up by the end of the week and add them to this repo's README. I hope the following gets you going. Let me know:

if name == "main": app.run(host='0.0.0.0', port='60050')


- adjust the port to something appropriate
- run the python script with your files in ```/static``` of whatever path you specify when you construct the ```app``

I run all of this out of Docker container, and will document that as well.   

I hope this quick description is helpful.   I will incorporate your judgement, and appropriate improvements, when I have a chance to do this later in the week.
genec1 commented 2 years ago

This is very much appreciated! I do not have the time to try it at the moment, but will give it a go in a week or two.

paul-shannon commented 2 years ago

@genec1 I think I have figured out a few changes needed to run igvR from a remote RStudio (or plain R) session. I will be testing in the next few days. I hope this will be useful to you.

paul-shannon commented 2 years ago

@genec1 igvR 1.15.8 and BrowserViz 2.17.1 just passed all tests for me, running in a remote RStudio Server session. Give it a try?

genec1 commented 2 years ago

@paul-shannon I can confirm that it works with the GitHub versions when using the host parameter and running on a remote host. Awesome!

paul-shannon commented 2 years ago

Glad this worked out! Let me know if you have other suggestions.

On Mar 1, 2022, at 4:39 PM, genec1 @.***> wrote:

@paul-shannon I can confirm that it works with the GitHub versions when using the host parameter and running on a remote host. Awesome!

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.