search

choishingwan / PRSice

A software package for calculating, applying, evaluating and plotting the results of polygenic risk scores
http://prsice.info
GNU General Public License v3.0
182 stars 86 forks source link

PRSice 2.2.4 error "munmap_chunk(): invalid pointer:" when calculating allele frequencies #128

Closed leevileh closed 5 years ago

leevileh commented 5 years ago

Hi,

I have a problem with PRSice 2.2.4 crashing when the program reaches the part "Calculate MAF and perform filtering on target SNPs". I'm getting the same problem also with the 2.2.2. version. Below is the log and error message I get. Any idea of what is going wrong?

Best regards, Leevi Lehtonen

Log:

PRSice 2.2.4 (18 July 2019) https://github.com/choishingwan/PRSice (C) 2016-2019 Shing Wan (Sam) Choi and Paul F. O'Reilly GNU General Public License v3

If you use PRSice in any published work, please cite: Choi SW, O'Reilly PF. PRSice-2: Polygenic Risk Score Software for Biobank-Scale Data. GigaScience 8, no. 7 (July 1, 2019)

2019-07-25 11:18:30 /path/user/newPRSice/PRSice/bin/PRSice \ --A1 ALLELE1 \ --A2 ALLELE0 \ --allow-inter \ --bar-levels 0.001,0.05,0.1,0.2,0.3,0.4,0.5,1 \ --base autosomes_mafPinfoFiltered_BOLTrun3pheno.assoc \ --beta \ --binary-target F \ --bp BP \ --chr CHR \ --clump-kb 500 \ --clump-p 1.000000 \ --clump-r2 0.100000 \ --extract PRSice.valid \ --hard-thres 0.100000 \ --interval 5e-05 \ --keep pheno174850_FID_IID.txt \ --lower 5e-08 \ --memory 19GB \ --missing MEAN_IMPUTE \ --model add \ --out PRSice \ --pheno-col scaled_pheno \ --pheno-file /path/user/pheno_bolt174850.txt \ --print-snp \ --pvalue P_BOLT_LMM \ --score avg \ --seed 1867599767 \ --snp SNP \ --stat BETA \ --target /path/geno_v3/ukb_imp_chr#_v3 \ --thread 8 \ --type bgen \ --upper 0.5

Initializing Genotype file: /path/geno_v3/ukb_imp_chr#_v3 (bgen)

Start processing autosomes_mafPinfoFiltered_BOLTrun3pheno ==================================================

SNP extraction/exclusion list contains 5 columns, will assume first column contains the SNP ID

Base file: autosomes_mafPinfoFiltered_BOLTrun3pheno.assoc 770446 variant(s) observed in base file, with: 107280 variant(s) excluded based on user input 663166 total variant(s) included from base file

Loading Genotype info from target ==================================================

174850 people (0 male(s), 0 ale(s)) observed 174850 founder(s) included

13959821 ambiguous variant(s) excluded 663166 variant(s) included

Calculate MAF and perform filtering on target SNPs ==================================================

Calculating allele frequencies: 0.00% Error in `/path/user/newPRSice/PRSice/bin/PRSice': munmap_chunk(): invalid pointer: 0x0000000012539a10 ======= Backtrace: ========= /lib64/libc.so.6(+0x7f5d4)[0x2b21214755d4] /lib64/libstdc++.so.6(_ZNSt13basic_filebufIcSt11char_traitsIcEE26_M_destroy_internal_bufferEv+0x1e)[0x2b2120c8df1e] /lib64/libstdc++.so.6(_ZNSt13basic_filebufIcSt11char_traitsIcEE5closeEv+0x57)[0x2b2120c8e3b7] /lib64/libstdc++.so.6(_ZNSt14basic_ofstreamIcSt11char_traitsIcEED1Ev+0x35)[0x2b2120c8e665] /path/user/newPRSice/PRSice/bin/PRSice(_ZN9BinaryGen19calc_freq_gen_interERKdS1_S1_bbbbP8Genotype+0x1333)[0x6e4e7d] /path/user/newPRSice/PRSice/bin/PRSice(_ZN8Genotype27calc_freqs_and_intermediateERKdS1_S1bbbbbR8ReporterPS+0xf6)[0x729f2c] /path/user/newPRSice/PRSice/bin/PRSice(main+0xa83)[0x6d2fd3] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b21214183d5] /path/user/newPRSice/PRSice/bin/PRSice[0x6d2489] ======= Memory map: ======== 00400000-008ea000 r-xp 00000000 00:33 3492542443 /path/user/newPRSice/PRSice/bin/PRSice 00ae9000-00aea000 r--p 004e9000 00:33 3492542443 /path/user/newPRSice/PRSice/bin/PRSice 00aea000-00aec000 rw-p 004ea000 00:33 3492542443 /path/user/newPRSice/PRSice/bin/PRSice 00aec000-00b6f000 rw-p 00000000 00:00 0 010a2000-126c3000 rw-p 00000000 00:00 0 [heap] 2b2120581000-2b21205a3000 r-xp 00000000 fd:00 101967891 /usr/lib64/ld-2.17.so 2b21205a3000-2b21205a5000 rw-p 00000000 00:00 0 2b21205b2000-2b21205b8000 rw-p 00000000 00:00 0 2b21207a2000-2b21207a3000 r--p 00021000 fd:00 101967891 /usr/lib64/ld-2.17.so 2b21207a3000-2b21207a4000 rw-p 00022000 fd:00 101967891 /usr/lib64/ld-2.17.so 2b21207a4000-2b21207a5000 rw-p 00000000 00:00 0 2b21207a5000-2b21207ba000 r-xp 00000000 fd:00 100665648 /usr/lib64/libz.so.1.2.7 2b21207ba000-2b21209b9000 ---p 00015000 fd:00 100665648 /usr/lib64/libz.so.1.2.7 2b21209b9000-2b21209ba000 r--p 00014000 fd:00 100665648 /usr/lib64/libz.so.1.2.7 2b21209ba000-2b21209bb000 rw-p 00015000 fd:00 100665648 /usr/lib64/libz.so.1.2.7 2b21209bb000-2b21209d2000 r-xp 00000000 fd:00 100820399 /usr/lib64/libpthread-2.17.so 2b21209d2000-2b2120bd1000 ---p 00017000 fd:00 100820399 /usr/lib64/libpthread-2.17.so 2b2120bd1000-2b2120bd2000 r--p 00016000 fd:00 100820399 /usr/lib64/libpthread-2.17.so 2b2120bd2000-2b2120bd3000 rw-p 00017000 fd:00 100820399 /usr/lib64/libpthread-2.17.so 2b2120bd3000-2b2120bd7000 rw-p 00000000 00:00 0 2b2120bd7000-2b2120cc0000 r-xp 00000000 fd:00 100665584 /usr/lib64/libstdc++.so.6.0.19 2b2120cc0000-2b2120ebf000 ---p 000e9000 fd:00 100665584 /usr/lib64/libstdc++.so.6.0.19 2b2120ebf000-2b2120ec7000 r--p 000e8000 fd:00 100665584 /usr/lib64/libstdc++.so.6.0.19 2b2120ec7000-2b2120ec9000 rw-p 000f0000 fd:00 100665584 /usr/lib64/libstdc++.so.6.0.19 2b2120ec9000-2b2120ede000 rw-p 00000000 00:00 0 2b2120ede000-2b2120fdf000 r-xp 00000000 fd:00 100665575 /usr/lib64/libm-2.17.so 2b2120fdf000-2b21211de000 ---p 00101000 fd:00 100665575 /usr/lib64/libm-2.17.so 2b21211de000-2b21211df000 r--p 00100000 fd:00 100665575 /usr/lib64/libm-2.17.so 2b21211df000-2b21211e0000 rw-p 00101000 fd:00 100665575 /usr/lib64/libm-2.17.so 2b21211e0000-2b21211f5000 r-xp 00000000 fd:00 102017359 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 2b21211f5000-2b21213f4000 ---p 00015000 fd:00 102017359 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 2b21213f4000-2b21213f5000 r--p 00014000 fd:00 102017359 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 2b21213f5000-2b21213f6000 rw-p 00015000 fd:00 102017359 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 2b21213f6000-2b21215b8000 r-xp 00000000 fd:00 100665535 /usr/lib64/libc-2.17.so 2b21215b8000-2b21217b8000 ---p 001c2000 fd:00 100665535 /usr/lib64/libc-2.17.so 2b21217b8000-2b21217bc000 r--p 001c2000 fd:00 100665535 /usr/lib64/libc-2.17.so 2b21217bc000-2b21217be000 rw-p 001c6000 fd:00 100665535 /usr/lib64/libc-2.17.so 2b21217be000-2b21217c3000 rw-p 00000000 00:00 0 2b2121db1000-2b21225c1000 rw-p 00000000 00:00 0 2b2131606000-2b2140e07000 rw-p 00000000 00:00 0 7ffd50234000-7ffd50256000 rw-p 00000000 00:00 0 [stack] 7ffd5035b000-7ffd5035d000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Error: Execution halted

choishingwan commented 5 years ago

This is likely due to a known problem with 2.2.4's compillation. I am now uploading 2.2.5, which should have solved this problem

leevileh commented 5 years ago

Hi,

With 2.2.5 I still get an error at the same place of the run, but now with the following error message:

Calculate MAF and perform filtering on target SNPs ==================================================

^MCalculating allele frequencies: 0.00% glibc detected path/user/newPRSice/PRSice_linux/PRSice_linux: malloc(): memory corruption: 0x000000000b10eda0 *** ======= Backtrace: ========= [0x5cad22] [0x5ccf4d] [0x5cdb7b] [0x4df758] [0x4d7175] [0x4d72d3] [0x406182] [0x4745e0] [0x471862] [0x41a101] [0x401eaf] [0x5aaa96] [0x403a15] ======= Memory map: ======== 00400000-006ba000 r-xp 00000000 00:33 554453928 path/user/newPRSice/PRSice_linux/PRSice_linux 008ba000-008c4000 rw-p 002ba000 00:33 554453928 path/user/newPRSice/PRSice_linux/PRSice_linux 008c4000-0094f000 rw-p 00000000 00:00 0 011bb000-101e7000 rw-p 00000000 00:00 0 [heap] 2ac520462000-2ac522673000 rw-p 00000000 00:00 0 2ac5370f7000-2ac54e0f8000 rw-p 00000000 00:00 0 7ffd01154000-7ffd01176000 rw-p 00000000 00:00 0 [stack] 7ffd0117c000-7ffd0117e000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Error: Execution halted

choishingwan commented 5 years ago

There might be a memory leak in the calculate frequency function, however, my test data was unable to show the problem. I will need to find a way to replicate this error for me to identify the root problem .

choishingwan commented 5 years ago

I am trying to reproduce this using my own UKBiobank data. Using only chromosome 1, 21 and 22, I cannot reproduce the error. If you don't mind, could you please re-run your script using --allow-inter but without --maf 0.05? This should allow me to roughly deduce the area where this problem locates.

leevileh commented 5 years ago

That's the way I was running it previously (here is my command):

Rscript /path/user/newPRSice/PRSice_linux/PRSice.R --dir /path/user/newPRSice/PRSice_linux --prsice /path/user/newPRSice/PRSice_linux/PRSice_linux --base autosomes_mafPinfoFiltered_BOLTrun3pheno.assoc --A1 ALLELE1 --A2 ALLELE0 --snp SNP --bp BP --chr CHR --beta --stat BETA --pvalue P_BOLT_LMM --type bgen --target /fs/projects/ukbb/geno_v3/ukb_imp_chr#_v3 --thread 8 --binary-target F --allow-inter --clump-kb 500 --clump-r2 0.100000 --keep pheno174850_FID_IID.txt --pheno /path/user/pheno_bolt174850.txt --pheno-col scaled_testo --print-snp --extract PRSice.valid --memory 19Gb

Did you perhaps mean the other way around, i.e. without --allow-inter and with --maf 0.05?

choishingwan commented 5 years ago

I'm trying to figure out which part of the function is causing the problem (as I can't reproduce the problem with my own copy of unfiltered UKB bgen file)

So Ideally, I would like to know if any of the following help to avoid the glibc error:

Rscript /path/user/newPRSice/PRSice_linux/PRSice.R \
    --dir /path/user/newPRSice/PRSice_linux \
    --prsice /path/user/newPRSice/PRSice_linux/PRSice_linux \
    --base autosomes_mafPinfoFiltered_BOLTrun3pheno.assoc \
     --A1 ALLELE1 \
    --A2 ALLELE0 \
    --beta --stat BETA \
    --pvalue P_BOLT_LMM \
    --type bgen \
    --target /fs/projects/ukbb/geno_v3/ukb_imp_chr#_v3 \
    --thread 8 \
    --binary-target F \
    --clump-kb 500 \
    --clump-r2 0.100000 \
    --keep pheno174850_FID_IID.txt \
    --pheno /path/user/pheno_bolt174850.txt \
    --pheno-col scaled_testo \
    --print-snp \
    --extract PRSice.valid \
    --memory 19Gb \
    --allow-inter
Rscript /path/user/newPRSice/PRSice_linux/PRSice.R \
    --dir /path/user/newPRSice/PRSice_linux \
    --prsice /path/user/newPRSice/PRSice_linux/PRSice_linux \
    --base autosomes_mafPinfoFiltered_BOLTrun3pheno.assoc \
     --A1 ALLELE1 \
    --A2 ALLELE0 \
    --beta --stat BETA \
    --pvalue P_BOLT_LMM \
    --type bgen \
    --target /fs/projects/ukbb/geno_v3/ukb_imp_chr#_v3 \
    --thread 8 \
    --binary-target F \
    --clump-kb 500 \
    --clump-r2 0.100000 \
    --keep pheno174850_FID_IID.txt \
    --pheno /path/user/pheno_bolt174850.txt \
    --pheno-col scaled_testo \
    --print-snp \
    --extract PRSice.valid \
    --memory 19Gb \
    --maf 0.05

and 

Rscript /path/user/newPRSice/PRSice_linux/PRSice.R \
    --dir /path/user/newPRSice/PRSice_linux \
    --prsice /path/user/newPRSice/PRSice_linux/PRSice_linux \
    --base autosomes_mafPinfoFiltered_BOLTrun3pheno.assoc \
     --A1 ALLELE1 \
    --A2 ALLELE0 \
    --beta --stat BETA \
    --pvalue P_BOLT_LMM \
    --type bgen \
    --target /fs/projects/ukbb/geno_v3/ukb_imp_chr#_v3 \
    --thread 8 \
    --binary-target F \
    --clump-kb 500 \
    --clump-r2 0.100000 \
    --keep pheno174850_FID_IID.txt \
    --pheno /path/user/pheno_bolt174850.txt \
    --pheno-col scaled_testo \
    --print-snp \
    --extract PRSice.valid \
    --memory 19Gb
leevileh commented 5 years ago

The clumping starts working when I don't have either of the options --allow-inter and --maf 0.05 in use. When I have --maf but not --allow-inter, the error message is the following:

^MCalculating allele frequencies: 0.00% glibc detected /path/user/newPRSice/PRSice_linux/PRSice_linux: malloc(): memory corruption: 0x000000000c1a6da0 *** ======= Backtrace: ========= [0x5cad22] [0x5ccf4d] [0x5cdb7b] [0x4df758] [0x4d7175] [0x4d72d3] [0x406182] [0x4745e0] [0x471862] [0x41a101] [0x401eaf] [0x5aaa96] [0x403a15] ======= Memory map: ======== 00400000-006ba000 r-xp 00000000 00:33 554453928 /path/user/newPRSice/PRSice_linux/PRSice_linux 008ba000-008c4000 rw-p 002ba000 00:33 554453928 /path/user/newPRSice/PRSice_linux/PRSice_linux 008c4000-0094f000 rw-p 00000000 00:00 0 02253000-1127f000 rw-p 00000000 00:00 0 [heap] 2b72fa14e000-2b72fc35f000 rw-p 00000000 00:00 0 2b7310de3000-2b7327de4000 rw-p 00000000 00:00 0 7ffc508b5000-7ffc508d7000 rw-p 00000000 00:00 0 [stack] 7ffc5099c000-7ffc5099e000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Error: Execution halted

When I have --allow-inter, but not --maf the error message is:

^MCalculating allele frequencies: 0.00% glibc detected path/user/newPRSice/PRSice_linux/PRSice_linux: malloc(): memory corruption: 0x000000000b10eda0 *** ======= Backtrace: ========= [0x5cad22] [0x5ccf4d] [0x5cdb7b] [0x4df758] [0x4d7175] [0x4d72d3] [0x406182] [0x4745e0] [0x471862] [0x41a101] [0x401eaf] [0x5aaa96] [0x403a15] ======= Memory map: ======== 00400000-006ba000 r-xp 00000000 00:33 554453928 path/user/newPRSice/PRSice_linux/PRSice_linux 008ba000-008c4000 rw-p 002ba000 00:33 554453928 path/user/newPRSice/PRSice_linux/PRSice_linux 008c4000-0094f000 rw-p 00000000 00:00 0 011bb000-101e7000 rw-p 00000000 00:00 0 [heap] 2ac520462000-2ac522673000 rw-p 00000000 00:00 0 2ac5370f7000-2ac54e0f8000 rw-p 00000000 00:00 0 7ffd01154000-7ffd01176000 rw-p 00000000 00:00 0 [stack] 7ffd0117c000-7ffd0117e000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Error: Execution halted

Anyway, I now have the program working, although the clumping seems to take a lot of time (perhaps since I can't use the --allow-inter). Thanks a lot for your help!

choishingwan commented 5 years ago

Can you try and test the latest version? That might (or might not) have helped with this problem

On Tue, 6 Aug 2019 at 8:56 AM, leevileh notifications@github.com wrote:

The clumping starts working when I don't have either of the options --allow-inter and --maf 0.05 in use. When I have --maf but not --allow-inter, the error message is the following:

^MCalculating allele frequencies: 0.00% glibc detected /path/user/newPRSice/PRSice_linux/PRSice_linux: malloc(): memory corruption: 0x000000000c1a6da0 *** ======= Backtrace: ========= [0x5cad22] [0x5ccf4d] [0x5cdb7b] [0x4df758] [0x4d7175] [0x4d72d3] [0x406182] [0x4745e0] [0x471862] [0x41a101] [0x401eaf] [0x5aaa96] [0x403a15] ======= Memory map: ======== 00400000-006ba000 r-xp 00000000 00:33 554453928 /path/user/newPRSice/PRSice_linux/PRSice_linux 008ba000-008c4000 rw-p 002ba000 00:33 554453928 /path/user/newPRSice/PRSice_linux/PRSice_linux 008c4000-0094f000 rw-p 00000000 00:00 0 02253000-1127f000 rw-p 00000000 00:00 0 [heap] 2b72fa14e000-2b72fc35f000 rw-p 00000000 00:00 0 2b7310de3000-2b7327de4000 rw-p 00000000 00:00 0 7ffc508b5000-7ffc508d7000 rw-p 00000000 00:00 0 [stack] 7ffc5099c000-7ffc5099e000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Error: Execution halted

When I have --allow-inter, but not --maf the error message is:

^MCalculating allele frequencies: 0.00% glibc detected path/user/newPRSice/PRSice_linux/PRSice_linux: malloc(): memory corruption: 0x000000000b10eda0 *** ======= Backtrace: ========= [0x5cad22] [0x5ccf4d] [0x5cdb7b] [0x4df758] [0x4d7175] [0x4d72d3] [0x406182] [0x4745e0] [0x471862] [0x41a101] [0x401eaf] [0x5aaa96] [0x403a15] ======= Memory map: ======== 00400000-006ba000 r-xp 00000000 00:33 554453928 path/user/newPRSice/PRSice_linux/PRSice_linux 008ba000-008c4000 rw-p 002ba000 00:33 554453928 path/user/newPRSice/PRSice_linux/PRSice_linux 008c4000-0094f000 rw-p 00000000 00:00 0 011bb000-101e7000 rw-p 00000000 00:00 0 [heap] 2ac520462000-2ac522673000 rw-p 00000000 00:00 0 2ac5370f7000-2ac54e0f8000 rw-p 00000000 00:00 0 7ffd01154000-7ffd01176000 rw-p 00000000 00:00 0 [stack] 7ffd0117c000-7ffd0117e000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Error: Execution halted

Anyway, I now have the program working, although the clumping seems to take a lot of time (perhaps since I can't use the --allow-inter). Thanks a lot for your help!

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/choishingwan/PRSice/issues/128?email_source=notifications&email_token=AAJTRYXHDBS5ZB7PZWEQ743QDFYHXA5CNFSM4IHZ5ZJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3VBJ6Q#issuecomment-518657274, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJTRYXGUJLNVASRLRA3JILQDFYHXANCNFSM4IHZ5ZJQ .

-- Dr Shing Wan Choi Postdoctoral Fellow Genetics and Genomic Sciences Icahn School of Medicine, Mount Sinai, NYC

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.