search

bulik / ldsc

LD Score Regression (LDSC)
GNU General Public License v3.0
652 stars 345 forks source link

MemoryError: When running LDSCORE analysis #426

Open chirrie opened 8 months ago

chirrie commented 8 months ago

I am getting memory error when generating LDscores;

./ldsc.py --bfile "/dataE/AWIGenGWAS/shared/imputed_data_plink/all_imputed_map_qc" --chunk-size 500000 --l2 --ld-wind-cm 1 --out Awi-gen-LDscores

Reading genotypes from /dataE/AWIGenGWAS/shared/imputed_data_plink/all_imputed After filtering, 13976041 SNPs remain Estimating LD Score. Traceback (most recent call last): File "./ldsc.py", line 620, in ldscore(args, log) File "./ldsc.py", line 316, in ldscore lN = geno_array.ldScoreVarBlocks(block_left, args.chunksize, annot=annot$ File "/home/chebii/Awi-gen/FG_raw/boltlmm/ldsc/ldscore/ldscore.py", line 125$ return self.corSumVarBlocks(block_left, c, func, snp_getter, annot) File "/home/chebii/Awi-gen/FG_raw/boltlmm/ldsc/ldscore/ldscore.py", line 189$ A = snp_getter(b) File "/home/chebii/Awi-gen/FG_raw/boltlmm/ldsc/ldscore/ldscore.py", line 397$ X = np.array(slice.decode(self._bedcode), dtype="float64").reshape((b, nru$ MemoryError

Analysis finished at Thu Mar 21 09:06:42 2024

aikedan commented 7 months ago

hello. how do you handle this problem in your research ? ? since I have this issue now. looking forward your reply.

aksarkar commented 6 months ago

@chirrie @aikedan Run ldsc on a machine with more memory available.

chirrie commented 6 months ago

I am running it on HPC with massive memory. I am not sure if it us an issue with mem

On Sun, 5 May 2024, 23:18 Abhishek Sarkar, @.***> wrote:

@chirrie https://github.com/chirrie @aikedan https://github.com/aikedan Run ldsc on a machine with more memory available.

— Reply to this email directly, view it on GitHub https://github.com/bulik/ldsc/issues/426#issuecomment-2094935553, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZTQ2VGSEBHHIWRM3747N3ZA2HXXAVCNFSM6AAAAABFA4ERUSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUHEZTKNJVGM . You are receiving this because you were mentioned.Message ID: @.***>

aksarkar commented 6 months ago

@chirrie Confirm with your HPC administrator that you are correctly requesting and receiving the requested amount of memory.

Also note that you should not change --chunk-size from the default value.