How to compute "Overall Rigor Score (mean)"?

RESQUE-Framework / website

The Research Quality Evaluation Scheme

https://resque-framework.github.io/website/

MIT License

2 stars 3 forks source link

How to compute "Overall Rigor Score (mean)"? #59

Open nicebread opened 7 months ago

nicebread commented 7 months ago

Option 1 (currently implemented): sum(scores across all papers) / sum(max across all papers)
Option 2: mean ( sum(scores per paper) / sum(max per paper) )

Option 1 implies a stronger weighting of papers with a higher max score Option 2 says: "Every paper is equally weighted, regardless of the max score".

I think option 2 more reflects the general philosophy of RESQUE.

angaertner commented 7 months ago

Does max per paper vary/depend on entries of the candidates (e.g., max is lower for non-empirical papers)? I also think option 2 (regardless of max score) is the better solution.