statgen / Minimac4

GNU General Public License v3.0
56 stars 18 forks source link

Segmentation fault After 'Imputation successful' #42

Closed jjfarrell closed 1 year ago

jjfarrell commented 3 years ago

This is an imputation on the UK Biobank data using minimac 1.0 on chr7. I am getting a similar error on other chromosomes. Any suggestions for this? This job is running with 28 cores and 196 GB of memory. So close to completion!

 Imputation successful (114771 seconds) !!!

 Appending chunk to final output VCF File :  minimac4_v3/chr7/ukbb.hg38.imputed.chr7.dose.vcf.gz
/var/spool/sge/scc-zm3/job_scripts/6840501: line 14: 149323 Segmentation fault      minimac4 --refHaps m3vcf_v3/adsp-5k_v3.chr$1.m3vcf.gz --haps ukbb.hg38/chr$1.hg38.phased.vcf.gz --allTypedSites --cpus $NSLOTS --format GT,DS,GP --meta --prefix minimac4_v3/chr$1/ukbb.hg38.imputed.chr$1
jonathonl commented 3 years ago

This is a known issue for very large sample sizes. You need to increase --printBuffer. I would try --printBuffer 2000000000.

jjfarrell commented 3 years ago

Thanks! I will give that a try.

jonathonl commented 3 years ago

Reopening since this eventually needs to be fixed and may help others in the meantime.

aokulabasile commented 1 year ago

I ran into the same seg fault issue when imputing into ~480K samples in UK Biobank. I am currently trying the --printBuffer 2000000000 option to see if this corrects the problem. I don't see this as a parameter option anywhere in the documentation. Can you please provide some more information on what printBuffer does, and on the reason for the seg fault issue with large datasets. Thank you!

jonathonl commented 1 year ago

@aokulabasile, which version of Minimac4 are you using? The printBuffer option referred to a temporary data buffer used for formatting VCF output. It no longer exists in versions >= 4.1.0. The segfault was caused by trying to format more dosages than the buffer could fit.

aokulabasile commented 1 year ago

Thanks for the quick response @jonathonl. I am using v1.0.2. Has this issue been corrected in versions ≥ 4.1.0? Thanks for your help!

jonathonl commented 1 year ago

Yes, this has been fixed in v4.1.0.

aokulabasile commented 1 year ago

Great, thank you for the help. Will run using the most recent version.