marcelm
commented
9 months ago It turns out good single-end accuracy does not necessarily imply good paired-end accuracy, so here is an optimization round only on paired-end data:
| k | s | l | u | accuracy |
|---|---|---|---|---|
| 22 | 18 | -2 | -2 | 86.833475 |
| 23 | 19 | -2 | -2 | 86.8255125 |
| 21 | 17 | -2 | -2 | 86.818425 |
| 20 | 16 | -2 | -2 | 86.7994125 (suggested) |
| 19 | 15 | -1 | -1 | 86.7724 |
| 23 | 19 | -3 | -3 | 86.76275 |
| 18 | 14 | -1 | -1 | 86.7576875 |
| 21 | 17 | -1 | -1 | 86.734175 |
| 22 | 18 | -3 | -3 | 86.732175 |
| 22 | 18 | -1 | -1 | 86.7238125 |
| 20 | 16 | -3 | 2 | 86.511525 (baseline) |
Now the question is which parameter combination to pick because it is going to be a tradeoff between SE and PE accuracy. (We could select parameters depending on whether SE or PE data is mapped, but I’d prefer not to go down that route at the moment. Also, we actually support mixed input (SE+PE).)
Because PE accuracy is maybe more important than SE, the following table ranks the parameter combinations by PE accuracy plus 25% of SE accuracy:
| k | s | l | u | PE | SE |
|---|---|---|---|---|---|
| 20 | 16 | -2 | -2 | 86.7994125 | 74.947225 |
| 19 | 15 | -1 | -1 | 86.7724 | 74.988925 |
| 23 | 19 | -3 | -3 | 86.76275 | 75.0243 |
| 18 | 14 | -1 | -1 | 86.7576875 | 74.9946 |
| 22 | 18 | -3 | -3 | 86.732175 | 75.003925 |
| 21 | 17 | -3 | -3 | 86.6811875 | 74.988875 |
| 18 | 14 | -1 | 0 | 86.6850375 | 74.8734 |
| 20 | 16 | -3 | -3 | 86.643825 | 74.936225 |
| 22 | 18 | -3 | -2 | 86.6519875 | 74.882975 |
| 21 | 17 | -3 | -2 | 86.6393125 | 74.891725 |
Picking $k=20$, $s=16$, $l=u=-2$ would improve SE accuracy by 0.3 and PE accuracy by 0.18. (And it’s similar to what we have now.)
ksahlin
As discussed in #272, it may be appropriate to add 75 as a new canonical read length.
I did an extensive search for a good set of parameters that improves accuracy over the existing one.
Reads of lengths 75 currently use the same parameters as reads of lengths 50, which are: $k=20$, $s=16$, $l=-3$, $u=2$. The accuracy we get using these parameters is our baseline.
I tested the following values for the parameters:
The script I wrote only tests single-end data. The assumption is that good single-end performance also means good paired-end performance. The script optimizes accuracy averaged over all four genomes, with one heuristic: It only accepts parameter combinations where the accuracy is better than the baseline for all four genomes. This speeds it up: When it tests a combination, it tests the genomes one by one and can stop as soon as the obtained accuracy is lower than the baseline. By starting with the smallest genome (Drosophila), often the bigger genomes don’t need to be tested at all.
The parameter combinations with $s=k-2$ performed really well, but this comes at a cost of increased memory usage, so I report results for both separately.
Here are the top 10 parameter combinations and the baseline for $s=k-4$:
Here are the top 10 parameter combinations if we allow $s=k-2$:
My feeling is that this additional increase is not worth the cost in memory, at least not for the default settings (sampling rate increases from 1/5 to 1/3, i.e., 67% more randstrobes).
I am running the above combinations on paired-end data at the moment and will report the results later.