a question about AMBER 2.0.21-beta

[ziyewang]

I installed AMBER 2.0.21-beta and my team member installed another version we found that the values of the Completeness (bp) and Purity (bp) in AMBER 2.0.21-beta are same as avg_completeness_per_bp and avg_purity_per_bp in the other version. Has the names of the metrics changed from that used in the paper? What's the difference between Average purity (bp) in AMBER 2.0.21-beta and avg_purity_per_bp in older version? Thanks.

[fernandomeyer]

Thanks for your question. Yes, the names changed, because there are different ways to describe the metrics, at least informally. One could argue that the word average implies computing a metric for each bin individually and then averaging, which is not the case for the metrics in the Quality for sample. However, one could argue that those are also an average of a complete sample, as in the paper. Average purity (bp) in AMBER 2.0.21-beta is the average of the purity computed for each bin invividually (corresponds to equation 6 in the paper). We added the (bp) to distinguish it from (seq), where true/false positives are bps and sequences, respectively. In the older AMBER, every metric used bps as unit of true/false positive, and using seq is new in AMBER 2. avg_purity_per_bp corresponds to equation 10 in the paper. Chances are we'll rename again the metrics in the non-beta version to be consistent with the paper.

[ziyewang]

Thnaks very much.------------------ Original ------------------From: Fernando Meyer notifications@github.comDate: Tue,Sep 3,2019 4:52 PMTo: CAMI-challenge/AMBER AMBER@noreply.github.comCc: ziyewang ziye_xf@163.com, Author author@noreply.github.comSubject: Re: [CAMI-challenge/AMBER] a question about AMBER 2.0.21-beta (#36)Thanks for your question. Yes, the names changed, because there are different ways to describe the metrics, at least informally. One could argue that the word average implies computing a metric for each bin individually and then averaging, which is not the case for the metrics in the Quality for sample. However, one could argue that those are also an average of a complete sample, as in the paper. Average purity (bp) in AMBER 2.0.21-beta is the average of the purity computed for each bin invividually (corresponds to equation 6 in the paper). We added the (bp) to distinguish it from (seq), where true/false positives are bps and sequences, respectively. In the older AMBER, every metric used bps as unit of true/false positive, and using seq is new in AMBER 2. avg_purity_per_bp corresponds to equation 10 in the paper. Chances are we'll rename again the metrics in the non-beta version to be consistent with the paper.

—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or mute the thread. [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/CAMI-challenge/AMBER/issues/36?email_source=notifications\u0026email_token=AJBTKY4IZKJ6EZFV5TANV2TQHYQTNA5CNFSM4ITCSK32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5XP46I#issuecomment-527367801", "url": "https://github.com/CAMI-challenge/AMBER/issues/36?email_source=notifications\u0026email_token=AJBTKY4IZKJ6EZFV5TANV2TQHYQTNA5CNFSM4ITCSK32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5XP46I#issuecomment-527367801", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]