The warning: IBSS algorithm did not converge in 100 iterations!

[Y-Isaac]

HI,

I'm coming across an issue that it would be helpful to have your insight on. I applied Polyfun+Susie on multiple phenotypes, and the majority of the results were normal, but one phenotype showed a little high frequency of warnings: "IBSS algorithm did not converge in 100 iterations!" (57 warnings/ 1419 loci). To save computational resources, I randomly sampled 10% of the samples and used their genetic data as a reference. Thus, I suspect there is an inconsistency between summary statistics and the LD matrix, as mentioned in the log file. However, when I tried using a completely consistent LD reference, the warning still persisted.

Therefore, I tried other methods: First, I included the parameter --susie-max-iter 200. Even with 200 iterations, the IBSS still did not converge. After checking the results, I found only a little differences(no substantial difference). I am considering increasing the iteration limit further (e.g., to 500), but intuitively, this does not seem like a reliable approach.

Second, I used HESS with the parameter --hess-min-h2 1e-4 and the default 100 iterations. The warning persisted, and the results only slightly differed from those obtained using Susie's built-in estimator (also with 100 iterations), but also no substantial difference. I also tried adding the parameter susie-resvar-init 0.9, but the results did not change.

Third, I used the susie-resvar-hess parameter, as you suggested in #56. This led to a new error:

[INFO] Excluding SNPs with heritability less than 1.0000e-04 from the HESS estimation [INFO] Average local SNP heritability estimated by modified HESS over 100 iterations: 3.3134e+00 [INFO] HESS estimated causal effect size variance: 3.3134e-01 [INFO] Residual variance using the HESS estimate: -2.3134e+00 Traceback (most recent call last): File "/public/home/P202306/software/polyfun/finemapper.py", line 1275, in df_finemap = finemap_obj.finemap(locus_start=args.start, locus_end=args.end, num_causal_snps=args.max_num_causal, File "/public/home/P202306/software/polyfun/finemapper.py", line 790, in finemap assert residual_var>=0 AssertionError

This seems to be due to the residual variance being less than zero. Now, I am eager to receive your guidance on how to solve this problem. Thank you in advance for your help!

[omerwe]

@Y-Isaac this error comes from SuSiE, so I can't address it in full. It seems like something is unusual in the LD in this region. This sometimes happens e.g. in long-range LD regions (like the HLA region).

I would try looking at Issue #176 in the SuSiE GitHub repo, and the diagnostics they refer to. If this still doesn't work, you can try opening an issue in the SuSiE repo.

Sorry I can't help more, hopefully this will push you in the right direction.

[Y-Isaac]

Thanks for your help!