Raters in the retroactive compensation system may use different systems or scales (e.g. one may rate everyone independently 0-100 based on how well they match some given criteria, while another assigns percentages summing to 100% that describe the desired outcome distribution).
Right now, ratings are simply averaged, meaning that someone rating people higher (like when using an independent 0-100 system) has more impact on the outcome averages than someone rating people lower (like when using a sum to 100% system).
To account for this, ratings should be normalized within each rater's attribute ratings before all being averaged together. This is easily accomplished by dividing each rater's attribute ratings by the sum of their ratings for that attribute, to convert each rating into a value out of 100.
Thanks to mOde from Kleomedes for bringing this up!
Raters in the retroactive compensation system may use different systems or scales (e.g. one may rate everyone independently 0-100 based on how well they match some given criteria, while another assigns percentages summing to 100% that describe the desired outcome distribution).
Right now, ratings are simply averaged, meaning that someone rating people higher (like when using an independent 0-100 system) has more impact on the outcome averages than someone rating people lower (like when using a sum to 100% system).
To account for this, ratings should be normalized within each rater's attribute ratings before all being averaged together. This is easily accomplished by dividing each rater's attribute ratings by the sum of their ratings for that attribute, to convert each rating into a value out of 100.
Thanks to mOde from Kleomedes for bringing this up!