search

pdhoff / amen

R package for analysis of network and dyadic data
27 stars 11 forks source link

ame() segfaults before sampling with large network #3

Closed stnorton closed 3 years ago

stnorton commented 3 years ago

I am trying to run ame() on a large weighted network, but the model always segfaults before burn-in begins. This has happened across 5 different tries, but I've only gotten an informative error once that seems to suggest it has something to do with the singular value decomposition in the multiplicative effects portion of the model.

The network has 44,007 nodes, resulting in an adjacency matrix that takes up ~16 GB in memory. Edges are weighted are are positive real values. I believe this is a function of the size of the model because the same model will begin sampling on a subgraph of 8000 nodes. The process that does not appear to be running out of memory - if I provide it with 1 TB of RAM, it never uses more than around 400 GB.

The ame() call that segfaults:

test <- ame(Y = model_adj, Xrow = rai, R=1, family='nrm', symmetric = TRUE, 
            burn = 1000, nscan=10000, odens=50, seed=1017, print=T, plot=F)

The error:

 *** caught segfault ***
address 0x7fa77e20aee8, cause 'memory not mapped'

Traceback:
 1: La.svd(x, nu, nv)
 2: svd(E)
 3: ame(Y = model_adj, Xrow = rai, R = 1, family = "nrm", symmetric = TRUE,     burn = 1000, nscan = 10000, odens = 50, seed = 1017, print = T,     plot = F)
An irrecoverable exception occurred. R is aborting now ...
/var/spool/slurmd/job20933418/slurm_script: line 30: 19909 Segmentation fault      Rscript amen_model.R

And the output from sessionInfo() - the CPU architecture is not the same on every node, so libopenblas may be compiled against a different architecture on different runs:

R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server 7.8 (Maipo)

Matrix products: default
BLAS/LAPACK: /nas/longleaf/apps/r/4.0.3/lib/libopenblas_haswellp-r0.3.5.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] amen_1.4.4      igraph_1.2.6    forcats_0.5.0   stringr_1.4.0   dplyr_1.0.6     purrr_0.3.4     readr_1.4.0     tidyr_1.1.2     tibble_3.1.2    ggplot2_3.3.3   tidyverse_1.3.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5        cellranger_1.1.0  pillar_1.6.1      compiler_4.0.3    dbplyr_2.0.0      tools_4.0.3       lattice_0.20-41   jsonlite_1.7.2    lubridate_1.7.9.2 lifecycle_1.0.0  
[11] gtable_0.3.0      pkgconfig_2.0.3   rlang_0.4.11      Matrix_1.2-18     reprex_0.3.0      cli_2.5.0         rstudioapi_0.13   DBI_1.1.0         haven_2.3.1       withr_2.4.2      
[21] xml2_1.3.2        httr_1.4.2        fs_1.5.0          generics_0.1.0    vctrs_0.3.8       hms_0.5.3         grid_4.0.3        tidyselect_1.1.1  glue_1.4.2        R6_2.5.0         
[31] fansi_0.5.0       readxl_1.3.1      modelr_0.1.8      magrittr_2.0.1    backports_1.2.1   scales_1.1.1      ellipsis_0.3.2    rvest_0.3.6       assertthat_0.2.1  colorspace_2.0-1 
[41] utf8_1.2.1        stringi_1.5.3     munsell_0.5.0     broom_0.7.5       crayon_1.4.1
pdhoff commented 3 years ago

Thanks for your interest in the 'amen' package. I suspect that the issue is with performing an svd on a matrix that large. You might try if you can get the R function 'svd' will work on your sociomatrix, without using the 'amen' package first.

That being said - 'amen' uses an iterative MCMC algorithm, and at every step some linear algebra will need to be done. I suspect the size of your sociomatrix will be too large for 'amen' to handle. You might try instead a moment-based approach.