[BUG] segfault when running umap in RestRserve endpoint

[alexvpickering]

Describe the bug

segfault occurs when running Seurat::RunUMAP from RestRserve endpoint

To Reproduce

For example:

library(RestRserve)
library(Seurat)
app = Application$new()

app$add_get(
  path = "/health", 
  FUN = function(.req, .res) {
    .res$set_body("OK")
  })

app$add_post(
  path = "/addone", 
  FUN = function(.req, .res) {

    data("pbmc_small")
    pbmc_small
    # Run UMAP map on first 5 PCs
    pbmc_small <- RunUMAP(object = pbmc_small, dims = 1:5)

    result = list(x = head(colnames(pbmc_small)))
    .res$set_content_type("application/json")
    .res$set_body(result)
  })

backend = BackendRserve$new()
backend$start(app, http_port = 8081)

curl -H "Content-Type: application/json" -d '{"x":10}' localhost:8081/addone

This causes a segfault in the R process:

Attaching SeuratObject
Attaching sp
{"timestamp":"2022-06-08 12:29:53.130500","level":"INFO","name":"Application","pid":2046976,"msg":"","context":{"http_port":8081,"endpoints":{"POST":"/addone","HEAD":"/health","GET":"/health"}}}
-- running Rserve in this R session (pid=2046976), 2 server(s) --
(This session will block until Rserve is shut down)
Warning: The default method for RunUMAP has changed from calling Python UMAP via reticulate to the R-native UWOT using the cosine metric
To use Python UMAP via reticulate, set umap.method to 'umap-learn' and metric to 'correlation'
This message will be shown once per session
12:30:58 UMAP embedding parameters a = 0.9922 b = 1.112
12:30:58 Read 80 rows and found 5 numeric columns
12:30:58 Using Annoy for neighbor search, n_neighbors = 30
12:30:58 Building Annoy index with metric = cosine, n_trees = 50
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
12:30:58 Writing NN index file to temp file /tmp/RtmpZ1oqvS/file1f426963de9d0d
12:30:58 Searching Annoy index using 1 thread, search_k = 3000
12:30:58 Annoy recall = 100%

 *** caught segfault ***
address 0x7f76088ceff8, cause 'memory not mapped'

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:

Code runs fine outside of RestRserve endpoint:

library(Seurat)
data("pbmc_small")
pbmc_small
# Run UMAP map on first 5 PCs
pbmc_small <- RunUMAP(object = pbmc_small, dims = 1:5)

Expected behavior

no segfault, endpoint returns data

Environment information

Please provide output of the sessionInfo() command.

> sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Fedora Linux 35 (Workstation Edition)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblas-r0.3.19.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] sp_1.5-0           SeuratObject_4.1.0 Seurat_4.1.1       RestRserve_1.1.1  

loaded via a namespace (and not attached):
  [1] nlme_3.1-157          spatstat.sparse_2.1-1 matrixStats_0.62.0   
  [4] RcppAnnoy_0.0.19      RColorBrewer_1.1-3    httr_1.4.3           
  [7] sctransform_0.3.3     tools_4.2.0           backports_1.4.1      
 [10] utf8_1.2.2            R6_2.5.1              irlba_2.3.5          
 [13] rpart_4.1.16          KernSmooth_2.23-20    uwot_0.1.11          
 [16] mgcv_1.8-40           rgeos_0.5-9           DBI_1.1.2            
 [19] lazyeval_0.2.2        colorspace_2.0-3      gridExtra_2.3        
 [22] tidyselect_1.1.2      compiler_4.2.0        progressr_0.10.1     
 [25] cli_3.3.0             plotly_4.10.0         scales_1.2.0         
 [28] checkmate_2.1.0       spatstat.data_2.2-0   lmtest_0.9-40        
 [31] ggridges_0.5.3        pbapply_1.5-0         goftest_1.2-3        
 [34] stringr_1.4.0         digest_0.6.29         spatstat.utils_2.3-1 
 [37] pkgconfig_2.0.3       htmltools_0.5.2       parallelly_1.32.0    
 [40] fastmap_1.1.0         htmlwidgets_1.5.4     rlang_1.0.2          
 [43] shiny_1.7.1           generics_0.1.2        zoo_1.8-10           
 [46] jsonlite_1.8.0        spatstat.random_2.2-0 ica_1.0-2            
 [49] dplyr_1.0.9           magrittr_2.0.3        patchwork_1.1.1      
 [52] Matrix_1.4-1          Rcpp_1.0.8.3          munsell_0.5.0        
 [55] fansi_1.0.3           abind_1.4-5           reticulate_1.25      
 [58] lifecycle_1.0.1       stringi_1.7.6         MASS_7.3-56          
 [61] Rtsne_0.16            plyr_1.8.7            grid_4.2.0           
 [64] parallel_4.2.0        listenv_0.8.0         promises_1.2.0.1     
 [67] ggrepel_0.9.1         crayon_1.5.1          deldir_1.0-6         
 [70] miniUI_0.1.1.1        lattice_0.20-45       cowplot_1.1.1        
 [73] splines_4.2.0         tensor_1.5            pillar_1.7.0         
 [76] igraph_1.3.1          Rserve_1.8-10         uuid_1.1-0           
 [79] spatstat.geom_2.4-0   future.apply_1.9.0    reshape2_1.4.4       
 [82] codetools_0.2-18      leiden_0.4.2          glue_1.6.2           
 [85] data.table_1.14.2     png_0.1-7             vctrs_0.4.1          
 [88] httpuv_1.6.5          polyclip_1.10-0       spatstat.core_2.4-4  
 [91] gtable_0.3.0          RANN_2.6.1            purrr_0.3.4          
 [94] tidyr_1.2.0           scattermore_0.8       future_1.26.1        
 [97] assertthat_0.2.1      ggplot2_3.3.6         mime_0.12            
[100] xtable_1.8-4          later_1.3.0           survival_3.3-1       
[103] viridisLite_0.4.0     tibble_3.1.7          cluster_2.1.3        
[106] globals_0.15.0        fitdistrplus_1.1-8    ellipsis_0.3.2       
[109] ROCR_1.0-11

Additional context

Add any other context about the problem here.

[dselivanov]

It's possible of the issues with OpenMP & fork (if I recall correctly annoy uses OpenMP). Try to disable OpenMP and re-run your code. I don't think RestRserve / Rserve can do anything about that.

[alexvpickering]

Thanks for the quick reply @dselivanov! Any ideas how to disable OpenMP?

[s-u]

@alexvpickering Seurat is not fork-safe. You cannot load it before forking, so simply move library(Seurat) into the function.

[alexvpickering]

That did it - thanks @s-u !!!