search

chanzuckerberg / idseq-workflows

Portable WDL workflows for IDseq production pipelines
https://idseq.net/
MIT License
31 stars 12 forks source link

short-read-mngs auto_benchmark: measure precision/recall wrt taxa truth sets #83

Closed mlin closed 3 years ago

mlin commented 3 years ago

@kislyuk @katrinakalantar Here's where we landed getting #79 to a solid checkpoint, with the precision/recall curves now rendered in the benchmarking jupyter notebook e.g.:

image

(These results look poor due to being run vs the viral-only databases in the quick CI tests; I'm collating all the full-size results now)

mlin commented 3 years ago

Here's the figure for idseq_bench_5 on the full-size databases -- as we'd expect/hope on this simple synthetic example, "perfect" results vs NT while the higher sensitivity vs NR can (but doesn't have to) generate a false-positive.

image