search

opencobra / memote

memote – the genome-scale metabolic model test suite
https://memote.readthedocs.io/
Apache License 2.0
125 stars 26 forks source link

fix: remove preliminary inversion of metrics #618

Closed ChristianLieven closed 5 years ago

ChristianLieven commented 5 years ago

I think it is much less confusing if the metrics of unscored tests reflect the title/ test description in their meaning e.g. SBML Level and Version checks that the lvl and ver fulfill a certain condition outcome = sbml_version[:2] >= (3, 1). The metric of that test float(outcome) shouldn't be the inverse of this.

Removing the inversion makes labeling consistent for all unscored tests.

ChristianLieven commented 5 years ago

@Midnighter and I agreed on keeping a unified meaning of a test's metric even though it may contradict its description and title. In simple terms: 0 = Good, 1 = Bad.

If a test is moved from the unscored to the scored section in the report the description will thus have to be adjusted accordingly. This is because on the scored side the metrices are inverted to mean 100% = Good, 0% = Bad.

Midnighter commented 5 years ago

If a test is moved from the unscored to the scored section in the report the description will thus have to be adjusted accordingly. This is because on the scored side the metrices are inverted to mean 100% = Good, 0% = Bad.

My point for not introducing this change is exactly that the score is automatically calculated in the frontend by inverting the metric. So at the moment the meaning of the metric is consistent (excluding mistakes) among all tests and only for some of them a score is calculated using that metric.