Hi!
I am currently using a large dataset (N ~80M SNPs). I have two questions.
Q1)
While the coord step ran ,with errors such as opening chrom 23,24,25 and 26, still gave the out put file. Is this issue ignorable?
88.46%"Unable to open object (object 'chrom_23' doesn't exist)"
Did not find chromosome 23 in SS dataset.
Continuing.
92.31%"Unable to open object (object 'chrom_24' doesn't exist)"
Did not find chromosome 24 in SS dataset.
Continuing.
96.15%"Unable to open object (object 'chrom_26' doesn't exist)"
Did not find chromosome 26 in SS dataset.
Continuing.
100.00%
/python/3.6.8/lib/python3.6/site-packages/numpy/lib/scimath.py:274: RuntimeWarning: divide by zero encountered in log
return nx.log(x)
Q2)
But in the gibbs phase, the first two f values (01 and 0.3) ran successfully, but threw an assertion error as described below.
=============================== [96mLDpred v. 1.0.10[0m ===============================
Applying LDpred with LD radius: 200
6188957 SNP effects were found
Calculating LDpred-inf weights
Starting LDpred gibbs with f=1.0000
4.55%
9.09%
13.64%
18.18%
22.73%
27.27%
31.82%
36.36%
40.91%
45.45%
50.00%
54.55%
59.09%
63.64%
68.18%
72.73%
77.27%
81.82%/.local/lib/python3.6/site-packages/ldpred/ld.py:365: ComplexWarning: Casting complex values to real discards the imaginary part
summary_dict[1.11]={'name':'Genome-wide (LDscore) estimated heritability:','value':'%0.4f'%gw_h2_ld_score_est}
.local/lib/python3.6/site-packages/ldpred/ld.py:366: ComplexWarning: Casting complex values to real discards the imaginary part
summary_dict[1.12]={'name':'Chi-square lambda (inflation statistic).','value':'%0.4f'%chi_square_lambda}
local/lib/python3.6/site-packages/ldpred/LDpred_inf.py:48: ComplexWarning: Casting complex values to real discards the imaginary part
updated_betas[start_i: stop_i] = sp.dot(A_inv n , beta_hats[start_i: stop_i]) # Adjust the beta_hats
local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:170: ComplexWarning: Casting complex values to real discards the imaginary part
curr_post_means[snp_i] = cd['hdmp_hdmpn'] postp res_beta_hat_i
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:180: ComplexWarning: Casting complex values to real discards the imaginary part
curr_betas[snp_i] = proposed_beta # UPDATE BETA
local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:55: ComplexWarning: Casting complex values to real discards the imaginary part
rv_scalars[:]=sampl_var_shrink_factor sp.sqrt((hdmp_hdmpn) * (1.0 / ldpred_n))
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:390: ComplexWarning: Casting complex values to real discards the imaginary part
f.write('%s %d %s %s %s %0.4e %0.4e\n' % (chrom, pos, sid, nt1, nt2, raw_beta, ldpred_beta))
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: overflow encountered in cdouble_scalars
postp = numerator / (numerator + d_const_b2_exp)
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: invalid value encountered in cdouble_scalars
postp = numerator / (numerator + d_const_b2_exp)
Traceback (most recent call last):
File "/python/3.6.8/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/python/3.6.8/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/.local/lib/python3.6/site-packages/ldpred/main.py", line 8, in
run.main()
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 337, in main
main_with_args(sys.argv[1:])
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 323, in main_with_args
LDpred_gibbs.main(p_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 422, in main
sampl_var_shrink_factor=1, verbose=p_dict['debug'], summary_dict=summary_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 328, in ldpred_genomewide
snp_lrld=snp_lrld, print_progress=False)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 165, in ldpred_gibbs
assert sp.isreal(postp), 'The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.'
AssertionError: The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.
Is this an error rising due to memory issue or LD radius?
Hi! I am currently using a large dataset (N ~80M SNPs). I have two questions.
Q1) While the coord step ran ,with errors such as opening chrom 23,24,25 and 26, still gave the out put file. Is this issue ignorable? 88.46%"Unable to open object (object 'chrom_23' doesn't exist)" Did not find chromosome 23 in SS dataset. Continuing.
92.31%"Unable to open object (object 'chrom_24' doesn't exist)" Did not find chromosome 24 in SS dataset. Continuing.
96.15%"Unable to open object (object 'chrom_26' doesn't exist)" Did not find chromosome 26 in SS dataset. Continuing.
100.00% /python/3.6.8/lib/python3.6/site-packages/numpy/lib/scimath.py:274: RuntimeWarning: divide by zero encountered in log return nx.log(x)
Q2) But in the gibbs phase, the first two f values (01 and 0.3) ran successfully, but threw an assertion error as described below.
=============================== [96mLDpred v. 1.0.10[0m ===============================
Applying LDpred with LD radius: 200 6188957 SNP effects were found Calculating LDpred-inf weights Starting LDpred gibbs with f=1.0000
4.55% 9.09% 13.64% 18.18% 22.73% 27.27% 31.82% 36.36% 40.91% 45.45% 50.00% 54.55% 59.09% 63.64% 68.18% 72.73% 77.27% 81.82% 86.36% 90.91% 95.45% 100.00% 100.00% Starting LDpred gibbs with f=0.3000
4.55% 9.09% 13.64% 18.18% 22.73% 27.27% 31.82% 36.36% 40.91% 45.45% 50.00% 54.55% 59.09% 63.64% 68.18% 72.73% 77.27% 81.82% 86.36% 90.91% 95.45% 100.00% 100.00% Starting LDpred gibbs with f=0.1000
4.55% 9.09% 13.64% 18.18% 22.73% 27.27% 31.82% 36.36% 40.91% 45.45% 50.00% 54.55% 59.09% 63.64% 68.18% 72.73% 77.27% 81.82%/.local/lib/python3.6/site-packages/ldpred/ld.py:365: ComplexWarning: Casting complex values to real discards the imaginary part summary_dict[1.11]={'name':'Genome-wide (LDscore) estimated heritability:','value':'%0.4f'%gw_h2_ld_score_est} .local/lib/python3.6/site-packages/ldpred/ld.py:366: ComplexWarning: Casting complex values to real discards the imaginary part summary_dict[1.12]={'name':'Chi-square lambda (inflation statistic).','value':'%0.4f'%chi_square_lambda} local/lib/python3.6/site-packages/ldpred/LDpred_inf.py:48: ComplexWarning: Casting complex values to real discards the imaginary part updated_betas[start_i: stop_i] = sp.dot(A_inv n , beta_hats[start_i: stop_i]) # Adjust the beta_hats local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:170: ComplexWarning: Casting complex values to real discards the imaginary part curr_post_means[snp_i] = cd['hdmp_hdmpn'] postp res_beta_hat_i .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:180: ComplexWarning: Casting complex values to real discards the imaginary part curr_betas[snp_i] = proposed_beta # UPDATE BETA local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:55: ComplexWarning: Casting complex values to real discards the imaginary part rv_scalars[:]=sampl_var_shrink_factor sp.sqrt((hdmp_hdmpn) * (1.0 / ldpred_n)) .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:390: ComplexWarning: Casting complex values to real discards the imaginary part f.write('%s %d %s %s %s %0.4e %0.4e\n' % (chrom, pos, sid, nt1, nt2, raw_beta, ldpred_beta)) .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: overflow encountered in cdouble_scalars postp = numerator / (numerator + d_const_b2_exp) .local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: invalid value encountered in cdouble_scalars postp = numerator / (numerator + d_const_b2_exp) Traceback (most recent call last): File "/python/3.6.8/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/python/3.6.8/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/.local/lib/python3.6/site-packages/ldpred/main.py", line 8, in
run.main()
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 337, in main
main_with_args(sys.argv[1:])
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 323, in main_with_args
LDpred_gibbs.main(p_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 422, in main
sampl_var_shrink_factor=1, verbose=p_dict['debug'], summary_dict=summary_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 328, in ldpred_genomewide
snp_lrld=snp_lrld, print_progress=False)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 165, in ldpred_gibbs
assert sp.isreal(postp), 'The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.'
AssertionError: The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.
Is this an error rising due to memory issue or LD radius?
Thanks in advance!!