Support for multiple assessment inputs

[costas80]

Consider the case where we have one assessment that is expected to be made over a set of inputs, each being scored separately. These inputs could also be other assessments.

As it stands in the CAV we can represent what the assessment refers to (using cav:assesses) and the criteria used for the assessment (via cccev:Criterion). If however for an assessment we want to express that multiple scored inputs are considered, how should this be done? One approach could be to define one cccev:Criterion per such input that will each be scored (via cav:Score). This seems conceptually wrong however given that we have one criterion that is applied multiple times for the resulting assessment. Putting this differently, a criterion should be agnostic of its specific input, i.e. its use in a given assessment. Extending this thinking, a scenario, its parts and the involved criteria should be able to stand independently of a specific assessment (consider it as representing your assessment methodology/approach/tool).

Considering the model of the CAV what seems promising is that although a cav:Score is assignedTo a cccev:Criterion, nothing prevents us from assigning multiple scores to the same criterion. What is missing is the ability to optionally state what each specific score refers to, i.e. its input. This could be achieved by adding a property cav:scores that could point to a generic RDF resource (i.e. anything could be an input) with cardinality [0..*].

Using this property we can effectively express a single criterion that is considered and scored multiple times for different inputs to produce the resulting assessment and its statements.

As a final note on this point, it could be interesting to consider whether a cav:Statement should be able to be made on specific input. Currently this is nicely flexible as a statement can optionally refer to a scenario, a part of a scenario or a specific criterion. We could also consider allowing a statement to also refer to a given cav:Score as [0..*].

[jseguraf]

Dear @costas80 , we would like you to clarify what you understand by the input of a Score. As we see it, this inputs could be reduced to what the EvaluationContext means and represents. In which case, the cardinality between Score and EvaluationContext could be modified to 0.. in alignment with your idea that one Criterion may have multiple Scores assigned which can be done from EvaluationContext to Criterion or in a more direct way, from Score to Criterion (modifiying the cardinality from [1] to []).

[costas80]

Dear @jseguraf , I disagree with your proposal to consider the EvaluationContext for this. The EvaluationContext in my view is part of the pre-existing setup/context for assessments that is unrelated to any specific inputs (i.e. resource instances upon which the assessment's scoring is based). The EvaluationContext would be used for example for instructions on how to evaluate a criterion or for something more specific like a formula (which we discussed would be supported).

As an example of this consider that we want to assess the quality of a given student class based on their exam outputs (i.e. an assessment considering other assessments as input). In this case you may have:

• A Criterion on the evaluation to consider. This criterion may be e.g. "Does the student's exam have a pass mark?".
• An optional EvaluationContext linked to the Criterion to state e.g. "The pass mark is 65%".
• When assessing the overall class we want to evaluate this criterion against each student's exam (assessment) and give a "Yes"/"No" score. If we have 10 exams we will have 10 scores (linked to the same single criterion), however we don't know (a) which are the exams considered as input and (b) what score was applied to each exam. We cannot link the individual exams to the EvaluationContext because (a) this should be agnostic of specific input and (b) we still lack the link between an exam and its assigned score. This would be resolved by using the cav:scores property.

I hope this clarifies and makes the case for my proposal.

[jseguraf]

The WG discusses/elaborates on the following:

• We have things that are defined ex-ante that exists outside specific assessments (Scenario, Evaluation Context and Criteria).
• The WG sees that for the evaluation of the criteria there are necessary information that need to be defined in the cccev:Criterion, if context agnostic or the cav:EvaluationContext if context specific.
• We need to have the Result as a class and not as a property, because we need to be able to assign multiple scores to one criterion and also to relate the score with other things (e.g. Assessment, Statement, etc.).

Agreement: The WG decided to remove the "result" property that linked the cav:Score to the cccev:Value, and in the class cav:Score a new attribute named "Value" has been added.

The WG agrees on "input" point. An object property "hasInput" has been added.