Should the individual measures be weighted?

[stanpinteTheSignallingCompany]

This question was raised in munich workshop.

https://github.com/openETCS/model-evaluation/blob/master/Munich_20130416_minutes/Minute%2020130416%20WP7%20evaluation%20matrix.pdf

[MariellePetitDoche]

This question is not clear for me :

• does it means that each individual criteria shall have a given weight ? in such a cas how to evaluate the weight ?
• does it means that a weight shall be given to the answers depending on the person who fill the template of criteria ? In my point of view, the criteria is just a technical tool. However, we can add a kind of confidence criteria for each person which fill a template (like in the paper review process).

[stanpinteTheSignallingCompany]

I think it should be further discussed, but I would say yes, each criteria should have a weight (maybe from 1 to 10), that would enable to compute a total score for the language/tool.

For instance, if the assessor put a 2 (on a 0 to 3 scale), and if the criteria is weighted 5, it would give 10 points.

I don't think that we need a confidence criteria. The weight is a scoring tool. Adding a confidence criteria (like I'm 25% sure of my rating for criteria x) would make it very complicated.

I look forward further discussing that subject.

[janWelte]

I think a weighted is needed, especially to seperated must have criterias from pure comfort criterias. (Certain limits can be added) I think a weighted is needed, especially to separated must have criteria from pure comfort criteria. (Certain limits can be added) Confidence criteria would make it very difficult to get a conclusive result. Therefore we shall make clear that every evaluator uses the same scale and has confidence in his judgment. Confidence criteria would make it very difficult to get a conclusive result. Therefore we shall make clear that every evaluater usesthe same scale and has confidence in his judgment.

[MariellePetitDoche]

Evaluation and decision done.