search

weizhouUMICH / SAIGE

GNU Lesser General Public License v3.0
187 stars 72 forks source link

pcg did not converge in step1 #392

Closed shimaomao26 closed 2 years ago

shimaomao26 commented 2 years ago

Hello,

I am running SAIGE in 100k individuals for binary traits. Every time I could find pcg did not converge in the log but the program did neither stop nor raise waring. In the paper it was said that in UKB it took < 10 iters to converge (Supplementary,Figure 12). Interestingly, the step2 result was good in my run, without obvious inflation, and showed high consistence with other Tools.

Does it matter? Should I make sure step1 pcg is converged?

Thank you very much!

Maomao

A piece of log:

pcg did not converge. You may increase maxiter number.
iter from getPCG1ofSigmaAndVector 500
t1_Rinv_1 is  4.766132e+15 
Nglmm  4.241419e+15
ytao0210 commented 2 years ago

hi, I met the same problem with you, but my task is not done yet. I ran SAIGE in 100k samples for binary traits with 32 threads in step1, but 24 hours have passed, and the task has not been completed, how long did it take you to run 100k individuals?

shimaomao26 commented 2 years ago

hi, I met the same problem with you, but my task is not done yet. I ran SAIGE in 100k samples for binary traits with 32 threads in step1, but 24 hours have passed, and the task has not been completed, how long did it take you to run 100k individuals?

Hi, in my case it's 1.5 days on 32 threads.

ytao0210 commented 2 years ago

Thank you! My task has finished today, which used 34hours with 32 threads.

weizhouUMICH commented 2 years ago

Hi @ytao0210 and @shimaomao26,

Thanks for reporting the issue! The converging time depends on several factors and may vary by phenotypes. The computation time generally is linear to the number of markers for GRM and slightly more than linear to the number of samples. PCG can failed to converge when GRM is singular. Can you please if there are twins and duplicated samples in your data?

Thanks, Wei

shimaomao26 commented 2 years ago

Dear Wei,

Thank you very much for the answer.

In my case the arrayed genotypes were plenty QCed before step1, and would not have the fully related samples. Besides, I tried to run SAIGE 0.38 (without LOCO) on the same (plink) data, but didn't see this warning anymore...

Best Maomao

v0.38, no LOCO

....
     user    system   elapsed 
81124.356   508.219  3877.540 
t_end - t_begin, fitting the NULL model took
     user    system   elapsed 
81063.622   381.378  3864.258 
Start estimating variance ratios

v0.44, adding LOCO

...
t_end_Get_Coef_LOCO - t_begin_Get_Coef_LOCO
     user    system   elapsed 
21441.274    45.086  2687.342 
pcg did not converge. You may increase maxiter number.
iter from getPCG1ofSigmaAndVector 500
t1_Rinv_1 is  6.21698e+15 
Nglmm  5.53254e+15 
      user     system    elapsed 
514686.177   1835.546  65103.890 
t_end - t_begin, fitting the NULL model took
      user     system    elapsed 
514639.806   1794.683  65081.518 
Start estimating variance ratios
weizhouUMICH commented 2 years ago

Hi @shimaomao26

We have just released a new version 1.0.0. It has computational efficiency improvements for both Step 1 and Step 2 for single-variant and set-based tests. We have created a new program github page https://github.com/saigegit/SAIGE with the documentation provided https://saigegit.github.io/SAIGE-doc/

The program will be maintained by multiple SAIGE developers there. Please feel free to try the version 1.0.0 and report issues if any.

Thanks! Wei