Error: Estimating residual variance failed: the estimated value is negative

[jcm6t]

Thanks for making the PolyFun software available, and supporting it with great documentation.

During a finemapper run, we got the following warning which turned into an exception. I'm guessing that we are on the low end of sample size for using this approach (n=11,800) and this represents a sample size issue. We are using the provided UK BB LD-files.

python3 $POLYFUNDIR/finemapper.py \ --ld LD_files/chr7_26000001_29000001 \ --sumstats chr7.res.prior.Z \ --n 11802 \ --chr 7 \ --start 26000001 \ --end 29000001 \ --method susie \ --max-num-causal 5 \ --out chr7.finemap1.gz

• Is this assumption correct or is there possibly something else happening ?
• Probably be helpful to have a more graceful handling of this situation

thanks, Joe

[WARNING] R[write to console]: Error in (function (bhat, shat, R, n, var_y, XtX, Xty, yty, X_colmeans = NA, : Estimating residual variance failed: the estimated value is negative

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/mnt/t1/projects/polyfun/finemapper.py", line 1267, in df_finemap = finemap_obj.finemap(locus_start=args.start, locus_end=args.end, num_causal_snps=args.max_num_causal, File "/mnt/t1/projects/polyfun/finemapper.py", line 851, in finemap susie_obj = self.susieR.susie_bhat( AttributeError: module 'susieR' has no attribute 'susie_bhat'

[jdblischak]

I doubt this is an issue with your sample size. This is likely due to a mismatch between the study population and the LD reference panel. PolyFun runs susie_suff_stat() directly because it assumes you are using insample LD, e.g. how the PolyFun paper used UKBB summary statistics with UKBB LD.

I recommend passing the flag --susie-resvar 1 to finemapper.py. This results in susie_suff_stat() getting called with the arguments residual_variance = 1 and estimate_residual_variance = FALSE, which are the settings used by susie_rss().

https://github.com/omerwe/polyfun/blob/594df4f31d798943f3c1cca864b74c805372e522/finemapper.py#L844-L845

https://github.com/stephenslab/susieR/blob/508fbfce737809aba691507b76cabf8c6a96d655/R/susie_rss.R#L200

[jcm6t]

John, Thanks for the pointer to the susie interface issues. There is also a discussion happening here: https://github.com/stephenslab/susieR/issues/162