search

CAMI-challenge / bbx-binning-evaluation-jd

A biobox for CAMI binning evaluation
1 stars 1 forks source link

unsupervised evaluation sometimes returns negative ari values #9

Open p-hofmann opened 8 years ago

p-hofmann commented 8 years ago

The 'hybrid' mode was used in the test, so taxonomic assignments were used as bin ids. But I can not imagine how that could cause negative values. Example results for ari: -0.24 and -2.29 All results should be between 0 and 1.

fungs commented 8 years ago

We need to check the formula, I guess. negative values should not be possible.

p-hofmann commented 8 years ago

The error seems to be here: t1 = 2 * row_pair_sum * col_pair_sum / total_pair_sum the division by 2 from 'total_pair_sum' cancels out only one of the division by 2 caused by 'row_pair_sum' and 'col_pair_sum'. This means t1 would be twice as big as it should be.

The calculation should be: t1 = row_pair_sum * col_pair_sum / total_pair_sum " 2* " was removed to cancel out the other division.

Can you confirm this?

fungs commented 8 years ago

I think you are right peter, the factor by two should not be in t1. That was probably a leftover when I re-arranged the formula for computer calculation.