weizhouUMICH
commented
3 years ago Hi @aliamin222,
Could you please share more log details before the error?
BTW, I've noticed that you specified 2300G, which is much higher than needed.
The memory usage for the Step 1 can be estimated as MN/(3210^9) Gb + memory for the phenotype file. M is the number of markers and N is the number of samples in the plink file
Fo example, for N=400000 samples, M - 93000, the memory usage will be slightly higher than 9Gb.
Thanks, Wei
aliamin222
Hello, I am using UK biobank 500k imputed data for a gene-based analysis. for traits with sample size > 340k (quantitative traits) or 230k (binary traits) I get below error:
caught segfault address 0x2aadb5269de4, cause 'memory not mapped' . . An irrecoverable exception occurred. R is aborting now ... /hpc/apps/current/saige/v0.38.app/step1_fitNULLGLMM.R: line 27: 12298 Segmentation fault
I am using Saige version 0.38. When I manually decrease the sample size (less than 340k or 230k) , I do not get this error. can you let me know why I get this error for large sample sizes please? below is my bash script. Thank you Ali
!/bin/bash
SBATCH --nodes=1
SBATCH --ntasks=1
SBATCH --cpus-per-task=16
SBATCH --mem=2900G
SBATCH -e /hpc/scratch/logs/S1_imp_quant-%j.err
SBATCH -o /hpc/scratch/logs/S1_imp_quant-%j.out
/bin/echo Running on:
hostname. Job name ${SLURM_JOB_NAME}. Job ID ${SLURM_JOB_ID}. Starting:datei=${SLURM_ARRAY_TASK_ID} Trait=$(head --lines=$i ${TraitFile} | tail -1 | awk '{print $1}') echo ${Trait}impdata=/hpc/scratch/imp_data resultspath=/hpc/scratch/imp_res programpath=/hpc/scratch/SAIGE/extdata mkdir ${resultspath}/${Trait}
/bin/echo Running on:
hostname/bin/echo Job name: ${SLURM_JOB_NAME} /bin/echo Job ID: ${SLURM_JOB_ID} /bin/echo Task ID: ${SLURM_ARRAY_TASK_ID}echo "" echo "Starting Step 1"
echo "Pheno: ${trait}"
echo "Pheno type: ${trait_type}"
echo
dateecho ""Rscript ${programpath}/step1_fitNULLGLMM.R \ --plinkFile=${impdata}/UKBB_chr13 \ --phenoFile=${impdata}/phenotype.txt \ --phenoCol=${Trait} \ --covarColList=ge_pc1,ge_pc2,ge_pc3,ge_pc4,ge_pc5,ge_pc6,ge_pc7,ge_pc8,ge_pc9,ge_pc10 \ --sampleIDColinphenoFile=IID \ --traitType=quantitative \ --invNormalize=FALSE \ --outputPrefix=${resultspath}/${Trait}/${Trait} \ --outputPrefix_varRatio=${resultspath}/${Trait}/${Trait}_cat \ --sparseGRMFile=${resultspath}/sparseGRM_relatednessCutoff_0.125_2000_randomMarkersUsed.sparseGRM.mtx \ --sparseGRMSampleIDFile=${resultspath}/sparseGRM_relatednessCutoff_0.125_2000_randomMarkersUsed.sparseGRM.mtx.sampleIDs.txt \ --nThreads=8 \ --LOCO=FALSE \ --skipModelFitting=FALSE \ --IsSparseKin=TRUE \ --isCateVarianceRatio=TRUE \ --IsOverwriteVarianceRatioFile=TRUE
/bin/echo Finished at:
date