ANGSD / angsd

Program for analysing NGS data.
230 stars 50 forks source link

Coding error in bootstrap for 2d realSFS #194

Closed nspope closed 5 years ago

nspope commented 5 years ago

There is a coding error in the bootstrapping of the 2d-SFS, within the function emStep2 (realSFS_optim.cpp line 211) ... where

inner[inc] = pre[inc]*gls[0]->mat[s][x]*gls[1]->mat[bootstrap[s]][y];

should instead be

inner[inc] = pre[inc]*gls[0]->mat[bootstrap[s]][x]*gls[1]->mat[bootstrap[s]][y];

e.g. the original GL is used for the first population while the bootstrapped GL is used for the second population. Thus the likelihood calculation is currently wrong, within bootstrap iterations of realSFS for 2d spectra (other dimensions are fine, as is the non-bootstrap SFS estimate). This may be responsible for the wacky spectra in issues #100 and #130.

Nate

nspope commented 5 years ago

... this is fixed by necessity in pull request #195 that adds support for contig bootstrapping to realSFS

nspope commented 5 years ago

Fixed with 7c73f06