Mean score should be computed & displayed separately for raw or scaled

[mxposed]

• [x] I have checked that this issue has not already been reported.
• [x] I have confirmed this bug exists on the main branch of openproblems.

On the website currently the Mean score is a mean of raw scores + scaled scores

[LuckyMD]

The mean score should be a mean of the scaled scores only. That's why the scaling was done. I think that is currently the case (at least spot-checking denoising suggests that it is correct).

[mxposed]

I'm sorry if I missed that or if it's already resolved, but I see this is currently the case on the website, and I checked the code for commits and I don't think it's been fixed

[LuckyMD]

Not sure I understand what you mean... spot checking the top performer for DR 5k PBMCs again (https://openproblems.bio/benchmarks/dimensionality_reduction/tenx_5k_pbmc/) confirms that the mean of the scaled values of the 10 metrics is 1.554, which is what is shown. Especially if you go on raw and see the RMSE is 66k, you can see that the mean does not take this value into account.

[LuckyMD]

Or do you mean that there should be a different mean score for raw scores? This is not the case by design as the mean is what is ranked by... and to calculate this mean you need the normalized metric outputs, which is what scaling is.

[mxposed]

Oh, I now see what you mean, sorry, my bad. Yes, it's a bit confusing that by default I see raw scores, but mean score of scaled values. Closing this, my apologies

[LuckyMD]

Good point though about seeing raw vs scaled scores as default. I think we just need a better "tooltip" or explanation of this on the website to clarify.