bvilhjal / ldpred

MIT License
95 stars 58 forks source link

AssertionError in gibbs sampling step #150

Open PradoVarathan opened 4 years ago

PradoVarathan commented 4 years ago

Hi! I am currently using a large dataset (N ~80M SNPs). I have two questions.

Q1) While the coord step ran ,with errors such as opening chrom 23,24,25 and 26, still gave the out put file. Is this issue ignorable? 88.46%"Unable to open object (object 'chrom_23' doesn't exist)" Did not find chromosome 23 in SS dataset. Continuing.

92.31%"Unable to open object (object 'chrom_24' doesn't exist)" Did not find chromosome 24 in SS dataset. Continuing.

96.15%"Unable to open object (object 'chrom_26' doesn't exist)" Did not find chromosome 26 in SS dataset. Continuing.

100.00% /python/3.6.8/lib/python3.6/site-packages/numpy/lib/scimath.py:274: RuntimeWarning: divide by zero encountered in log return nx.log(x)

Q2) But in the gibbs phase, the first two f values (01 and 0.3) ran successfully, but threw an assertion error as described below.

=============================== LDpred v. 1.0.10 ===============================

Applying LDpred with LD radius: 200 6188957 SNP effects were found Calculating LDpred-inf weights Starting LDpred gibbs with f=1.0000

4.55% 9.09% 13.64% 18.18% 22.73% 27.27% 31.82% 36.36% 40.91% 45.45% 50.00% 54.55% 59.09% 63.64% 68.18% 72.73% 77.27% 81.82% 86.36% 90.91% 95.45% 100.00% 100.00% Starting LDpred gibbs with f=0.3000

4.55% 9.09% 13.64% 18.18% 22.73% 27.27% 31.82% 36.36% 40.91% 45.45% 50.00% 54.55% 59.09% 63.64% 68.18% 72.73% 77.27% 81.82% 86.36% 90.91% 95.45% 100.00% 100.00% Starting LDpred gibbs with f=0.1000

4.55% 9.09% 13.64% 18.18% 22.73% 27.27% 31.82% 36.36% 40.91% 45.45% 50.00% 54.55% 59.09% 63.64% 68.18% 72.73% 77.27% 81.82%/.local/lib/python3.6/site-packages/ldpred/ld.py:365: ComplexWarning: Casting complex values to real discards the imaginary part summary_dict[1.11]={'name':'Genome-wide (LDscore) estimated heritability:','value':'%0.4f'%gw_h2_ld_score_est} .local/lib/python3.6/site-packages/ldpred/ld.py:366: ComplexWarning: Casting complex values to real discards the imaginary part summary_dict[1.12]={'name':'Chi-square lambda (inflation statistic).','value':'%0.4f'%chi_square_lambda} local/lib/python3.6/site-packages/ldpred/LDpred_inf.py:48: ComplexWarning: Casting complex values to real discards the imaginary part updated_betas[start_i: stop_i] = sp.dot(A_inv n , beta_hats[start_i: stop_i]) # Adjust the beta_hats local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:170: ComplexWarning: Casting complex values to real discards the imaginary part curr_post_means[snp_i] = cd['hdmp_hdmpn'] postp res_beta_hat_i .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:180: ComplexWarning: Casting complex values to real discards the imaginary part curr_betas[snp_i] = proposed_beta # UPDATE BETA local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:55: ComplexWarning: Casting complex values to real discards the imaginary part rv_scalars[:]=sampl_var_shrink_factor sp.sqrt((hdmp_hdmpn) * (1.0 / ldpred_n)) .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:390: ComplexWarning: Casting complex values to real discards the imaginary part f.write('%s %d %s %s %s %0.4e %0.4e\n' % (chrom, pos, sid, nt1, nt2, raw_beta, ldpred_beta)) .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: overflow encountered in cdouble_scalars postp = numerator / (numerator + d_const_b2_exp) .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: invalid value encountered in cdouble_scalars postp = numerator / (numerator + d_const_b2_exp) Traceback (most recent call last): File "/python/3.6.8/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/python/3.6.8/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/.local/lib/python3.6/site-packages/ldpred/main.py", line 8, in run.main() File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 337, in main main_with_args(sys.argv[1:]) File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 323, in main_with_args LDpred_gibbs.main(p_dict) File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 422, in main sampl_var_shrink_factor=1, verbose=p_dict['debug'], summary_dict=summary_dict) File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 328, in ldpred_genomewide snp_lrld=snp_lrld, print_progress=False) File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 165, in ldpred_gibbs assert sp.isreal(postp), 'The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.' AssertionError: The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.

Is this an error rising due to memory issue or LD radius?

Thanks in advance!!