Running Dyno on HPC Cluster

[cohenp05]

Hi all,

Thanks for an amazing package. Forgive me as I'm new to using containers, so I'm having some trouble getting the singularity containers to run on our scientific computing cluster. I have pulled all the TI method images to my local machine and then uploaded them to the cluster, but I can't get singularity to recognize that directory as the cache directory to run out of. When I use the babelwhale call as suggested (like below)

##Specify where to place singularity containers
singularity_config <- dynwrap::set_default_config(dynwrap::create_singularity_config(cache_dir = "/hpc/users/cohenp05/Comp1_Container/"))
babelwhale::set_default_config(singularity_config)

and point the singularity cache_dir to my directory, I get this error:

FATAL:   Unable to handle docker://alpine:3.7 uri: failed to get SHA of docker://alpine:3.7: pinging docker registry returned: Get https://registry-1.docker.io/v2/: dial tcp 52.22.67.152:443: connect: network is unreachable

Here is my sessionInfo:

R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /hpc/packages/minerva-centos7/intel/parallel_studio_xe_2019/compilers_and_libraries_2019.0.117/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so

locale:
[1] C

attached base packages:
 [1] splines   stats4    parallel  stats     graphics  grDevices utils    
 [8] datasets  methods   base     

other attached packages:
 [1] rlist_0.4.6.1       forcats_0.4.0       stringr_1.4.0      
 [4] purrr_0.3.2         readr_1.3.1         tidyr_0.8.3        
 [7] tibble_2.1.1        tidyverse_1.2.1     dyno_0.1.1         
[10] dynwrap_1.1.3       dynplot_1.0.1       dynmethods_1.0.4   
[13] dynguidelines_1.0.0 dynfeature_1.0.0    RColorBrewer_1.1-2 
[16] data.table_1.12.2   monocle_2.10.1      DDRTree_0.1.5      
[19] irlba_2.3.3         VGAM_1.1-1          Biobase_2.42.0     
[22] BiocGenerics_0.28.0 Matrix_1.2-15       gridExtra_2.3      
[25] Seurat_3.0.2        dplyr_0.8.1         plyr_1.8.4         
[28] cowplot_0.9.4       ggplot2_3.1.1      

loaded via a namespace (and not attached):
  [1] reticulate_1.12       R.utils_2.8.0         tidyselect_0.2.5     
  [4] htmlwidgets_1.3       grid_3.5.3            combinat_0.0-8       
  [7] ranger_0.11.2         docopt_0.6.1          Rtsne_0.15           
 [10] munsell_0.5.0         codetools_0.2-16      ica_1.0-2            
 [13] future_1.13.0         withr_2.1.2           colorspace_1.4-1     
 [16] fastICA_1.2-1         knitr_1.23            rstudioapi_0.10      
 [19] ROCR_1.0-7            gbRd_0.4-11           listenv_0.7.0        
 [22] Rdpack_0.11-0         slam_0.1-45           polyclip_1.10-0      
 [25] farver_1.1.0          pheatmap_1.0.12       generics_0.0.2       
 [28] xfun_0.7              R6_2.4.0              GA_3.2               
 [31] rsvd_1.0.0            pdist_1.2             bitops_1.0-6         
 [34] assertthat_0.2.1      promises_1.0.1        SDMTools_1.1-221.1   
 [37] scales_1.0.0          ggraph_1.0.2          nnet_7.3-12          
 [40] gtable_0.3.0          babelwhale_0.0.0.9000 npsurv_0.4-0         
 [43] globals_0.12.4        processx_3.3.1        tidygraph_1.1.2      
 [46] rlang_0.3.4           lazyeval_0.2.2        acepack_1.4.1        
 [49] broom_0.5.2           checkmate_1.9.3       yaml_2.2.0           
 [52] reshape2_1.4.3        modelr_0.1.4          backports_1.1.4      
 [55] httpuv_1.5.1          Hmisc_4.2-0           tools_3.5.3          
 [58] gplots_3.0.1.1        ggridges_0.5.1        Rcpp_1.0.1           
 [61] base64enc_0.1-3       densityClust_0.3      ps_1.3.0             
 [64] rpart_4.1-15          pbapply_1.4-0         viridis_0.5.1        
 [67] dynparam_1.0.0        zoo_1.8-5             haven_2.1.0          
 [70] ggrepel_0.8.1         cluster_2.0.7-1       magrittr_1.5         
 [73] carrier_0.1.0         lmtest_0.9-37         RANN_2.6.1           
 [76] fitdistrplus_1.0-14   matrixStats_0.54.0    hms_0.4.2            
 [79] patchwork_0.0.1       lsei_1.2-0            mime_0.6             
 [82] xtable_1.8-4          readxl_1.3.1          sparsesvd_0.1-4      
 [85] HSMMSingleCell_1.2.0  testthat_2.1.1        compiler_3.5.3       
 [88] KernSmooth_2.23-15    crayon_1.3.4          rje_1.9              
 [91] R.oo_1.22.0           htmltools_0.3.6       proxyC_0.1.4         
 [94] later_0.8.0           Formula_1.2-3         RcppParallel_4.4.2   
 [97] lubridate_1.7.4       tweenr_1.0.1          MASS_7.3-51.1        
[100] cli_1.1.0             R.methodsS3_1.7.1     gdata_2.18.0         
[103] metap_1.1             igraph_1.2.4.1        pkgconfig_2.0.2      
[106] foreign_0.8-71        plotly_4.9.0          xml2_1.2.0           
[109] foreach_1.4.4         dynutils_1.0.3        rvest_0.3.4          
[112] bibtex_0.4.2          digest_0.6.19         dyndimred_1.0.1      
[115] sctransform_0.2.0     tsne_0.1-3            cellranger_1.1.0     
[118] htmlTable_1.13.1      shiny_1.3.2           gtools_3.8.1         
[121] nlme_3.1-137          jsonlite_1.6          viridisLite_0.3.0    
[124] limma_3.38.3          pillar_1.4.0          lattice_0.20-38      
[127] httr_1.4.0            survival_2.44-1.1     glue_1.3.1           
[130] remotes_2.0.4         qlcMatrix_0.9.7       FNN_1.1.3            
[133] png_0.1-7             iterators_1.0.10      ggforce_0.2.2        
[136] stringi_1.4.3         latticeExtra_0.6-28   caTools_1.17.1.2     
[139] future.apply_1.2.0    ape_5.3

I've also tried running the containers in a bash script by exporting my dynverse object and a dimensional reduction object to h5 files and running singularity run directly on the image like so:

ml singularity
module load R/3.5.3
SINGULARITY_CACHEDIR="/hpc/users/cohenp05/Comp1_Container/"
export SINGULARITY_CACHEDIR

singularity cache list

singularity run /hpc/users/cohenp05/Comp1_Container/ti_comp1_latest.sif --dataset=/hpc/users/cohenp05/Dyno_Objects/Monocyte_Dyno.h5 --output=/hpc/users/cohenp05/Dyno_Objects/Monocyte_Comp1.h5 --dimred=/hpc/users/cohenp05/Dyno_Objects/Monocytes_UMAP.h5

And then I get this error

Error: --dataset should contain a pathname of a .loom or .h5 file. Add a '-h' flag for help.
Execution halted

Please help! I'm out of ideas.

[rcannood]

Hi @cohenp05 !

It seems like trying to run it manually might be our first priority to try to find out what's going wrong.

Unfortunately, you can't directly pass files into a singularity or docker container. You need to mount the folder into the container, and then specify the path of the dataset in the new location. THe nice thing is that it should simplify your command a little:

singularity run -B /hpc/users/cohenp05/Dyno_Objects:/mount /hpc/users/cohenp05/Comp1_Container/ti_comp1_latest.sif --dataset=/mount/Monocyte_Dyno.h5 --output=/mount/Monocyte_Comp1.h5 --dimred=umap

Also, if you look at singularity run /hpc/users/cohenp05/Comp1_Container/ti_comp1_latest.sif --help, it should let you know that the dimred parameter can only be used as a string specifying which dimred method to use -- you can't provide your own dimred this way.

Could you verify whether running it manually succeeds?

Robrecht

[rcannood]

Closing this issue. Feel free to reply if further input is needed.