privefl / bigsnpr

R package for the analysis of massive SNP arrays.
https://privefl.github.io/bigsnpr/
196 stars 44 forks source link

Invalid pointer with `snp_ldsplit` #508

Closed biona001 closed 4 months ago

biona001 commented 4 months ago

I'm running bigsnpr on the cluster. MWE:

  1. First I module load R/4.2.2 gcc/7.5.0 intel/2020.4
  2. Then I start R and run the following
    
    library("bigsnpr")
    plinkfile = "/u/home/b/biona001/project-loes/ghostknockoff/GLIMPSE2_QC/chr22.bed"
    fbmfile = "/u/home/b/biona001/project-loes/ghostknockoff/GLIMPSE2_QC/FBM/chr22"

import PLINK data as FBM (file backed matrix) format

rdsfile <- paste0(fbmfile, ".rds") if (!file.exists(rdsfile)){snp_readBed2(plinkfile, backingfile = fbmfile)} x <- snp_attach(rdsfile)

estimate correlation matrix

corr <- snp_cor(x$genotypes, ind.col=1:1000, infos.pos=x$map$physical.pos[1:1000], ncores=1)

compute LD regions

THR_R2 <- 0.01 m <- ncol(corr) SEQ <- round(seq_log(m / 30, m / 5, length.out = 10)) res <- snp_ldsplit(corr, thr_r2 = THR_R2, min_size = 100, max_size = SEQ)

The last step throws an error

res <- snp_ldsplit(corr, thr_r2 = THR_R2, min_size = 100, max_size = SEQ) Error in `/u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/bin/exec/R': free(): invalid pointer: 0x0000000004e30f28 ======= Backtrace: ========= /lib64/libc.so.6(+0x81329)[0x2b92770d0329] /u/home/b/biona001/R/x86_64-pc-linux-gnu-library/4.2.2/bigsnpr/libs/bigsnpr.so(_Z31exception_to_condition_templateISt9exceptionEP7SEXPRECRKT_b+0x301)[0x2b9289075ea1] /u/home/b/biona001/R/x86_64-pc-linux-gnu-library/4.2.2/bigsnpr/libs/bigsnpr.so(_bigsnpr_get_C+0x1cb)[0x2b928907339b] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x1014a4)[0x2b927663f4a4] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x14558d)[0x2b927668358d] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x178)[0x2b9276691118] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x15496b)[0x2b927669296b] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_applyClosure+0x1c1)[0x2b92766936f1] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x14324a)[0x2b927668124a] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x178)[0x2b9276691118] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x15496b)[0x2b927669296b] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_applyClosure+0x1c1)[0x2b92766936f1] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(R_forceAndCall+0x17b)[0x2b927669607b] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x86e3c)[0x2b92765c4e3c] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x1976f2)[0x2b92766d56f2] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x143512)[0x2b9276681512] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x178)[0x2b9276691118] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x15496b)[0x2b927669296b] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_applyClosure+0x1c1)[0x2b92766936f1] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x14324a)[0x2b927668124a] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x178)[0x2b9276691118] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x153a8e)[0x2b9276691a8e] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x1529d2)[0x2b92766909d2] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x178)[0x2b9276691118] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x15496b)[0x2b927669296b] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_applyClosure+0x1c1)[0x2b92766936f1] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x14324a)[0x2b927668124a] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x178)[0x2b9276691118] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x15496b)[0x2b927669296b] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_applyClosure+0x1c1)[0x2b92766936f1] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x2ae)[0x2b927669124e] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x157a8a)[0x2b9276695a8a] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_eval+0x51f)[0x2b92766914bf] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(Rf_ReplIteration+0x252)[0x2b92766c6152] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(+0x1884c1)[0x2b92766c64c1] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so(run_Rmainloop+0x48)[0x2b92766c6558] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/bin/exec/R(main+0x1b)[0x40075b] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b9277071555] /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/bin/exec/R[0x40078b] ======= Memory map: ======== 00400000-00401000 r-xp 00000000 00:34 6317251420151053054 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/bin/exec/R 00600000-00601000 r--p 00000000 00:34 6317251420151053054 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/bin/exec/R 00601000-00602000 rw-p 00001000 00:34 6317251420151053054 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/bin/exec/R 01cb9000-0f10f000 rw-p 00000000 00:00 0 [heap] 2b927631a000-2b927633c000 r-xp 00000000 08:03 1331089 /usr/lib64/ld-2.17.so 2b927633c000-2b927633f000 rw-p 00000000 00:00 0 2b927633f000-2b9276340000 r--p 00000000 00:34 2254763068395539482 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/library/translations/en/LC_MESSAGES/R.mo 2b9276340000-2b9276341000 rw-p 00000000 00:00 0 2b9276341000-2b9276342000 -w-s 00000000 08:04 9306115 /work/tmp/RtmpRZ6oFn/file602eabd8a2.bk 2b9276342000-2b9276349000 r--s 00000000 08:03 1716066 /usr/lib64/gconv/gconv-modules.cache 2b9276349000-2b927634a000 rw-p 00000000 00:00 0 2b9276361000-2b92764b3000 rw-p 00000000 00:00 0 2b927653b000-2b927653c000 r--p 00021000 08:03 1331089 /usr/lib64/ld-2.17.so 2b927653c000-2b927653d000 rw-p 00022000 08:03 1331089 /usr/lib64/ld-2.17.so 2b927653d000-2b927653e000 rw-p 00000000 00:00 0 2b927653e000-2b9276871000 r-xp 00000000 00:34 7697395682123609363 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so 2b9276871000-2b9276a70000 ---p 00333000 00:34 7697395682123609363 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so 2b9276a70000-2b9276a8e000 r--p 00332000 00:34 7697395682123609363 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so 2b9276a8e000-2b9276a9d000 rw-p 00350000 00:34 7697395682123609363 /u/local/apps/R/4.2.2/gcc-4.8.5/intel-2020.4/lib64/R/lib/libR.so 2b9276a9d000-2b9276c05000 rw-p 00000000 00:00 0 2b9276c05000-2b9276c32000 r-xp 00000000 00:34 7890178973220366635 /u/local/compilers/gcc/7.5.0/lib64/libgomp.so.1.0.0 2b9276c32000-2b9276e31000 ---p 0002d000 00:34 7890178973220366635 /u/local/compilers/gcc/7.5.0/lib64/libgomp.so.1.0.0 2b9276e31000-2b9276e32000 r--p 0002c000 00:34 7890178973220366635 /u/local/compilers/gcc/7.5.0/lib64/libgomp.so.1.0.0 2b9276e32000-2b9276e33000 rw-p 0002d000 00:34 7890178973220366635 /u/local/compilers/gcc/7.5.0/lib64/libgomp.so.1.0.0 2b9276e33000-2b9276e4a000 r-xp 00000000 08:03 1330479 /usr/lib64/libpthread-2.17.so 2b9276e4a000-2b9277049000 ---p 00017000 08:03 1330479 /usr/lib64/libpthread-2.17.so 2b9277049000-2b927704a000 r--p 00016000 08:03 1330479 /usr/lib64/libpthread-2.17.so 2b927704a000-2b927704b000 rw-p 00017000 08:03 1330479 /usr/lib64/libpthread-2.17.so 2b927704b000-2b927704f000 rw-p 00000000 00:00 0 2b927704f000-2b9277213000 r-xp 00000000 08:03 1331231 /usr/lib64/libc-2.17.so 2b9277213000-2b9277412000 ---p 001c4000 08:03 1331231 /usr/lib64/libc-2.17.so 2b9277412000-2b9277416000 r--p 001c3000 08:03 1331231 /usr/lib64/libc-2.17.soAborted


Any idea what could be happening?
biona001 commented 4 months ago

Sorry, I just realized I opened this issue under bigstatsr rather than bigsnpr. Let me know if you want me to close this and reopen it under the right repo.

privefl commented 4 months ago

Thanks for reporting. I've never encountered this issue before.

biona001 commented 4 months ago

Hi @privefl, thank you for the quick response.

R version 4.2.2 (2022-10-31) -- "Innocent and Trusting" Copyright (C) 2022 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.

library("bigsnpr") Loading required package: bigstatsr Error: package or namespace load failed for ‘bigstatsr’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/u/home/b/biona001/R/x86_64-pc-linux-gnu-library/4.2.2/bigstatsr/libs/bigstatsr.so': /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /u/home/b/biona001/R/x86_64-pc-linux-gnu-library/4.2.2/bigstatsr/libs/bigstatsr.so) Error: package ‘bigstatsr’ could not be loaded

biona001 commented 4 months ago

@privefl I just tested the same code on the same data, but on a different cluster. Here I only module load R/4.0.2 and I am no longer getting this invalid pointer issue. Thus, I think the issue is indeed due to environment variables being set incorrectly. I'd be fine if you close this issue.

privefl commented 4 months ago

I guess it is something that has to do with the way the package has been installed on the cluster. You should contact the IT team responsible of the cluster to get some help.