PetervDooren
commented
2 weeks ago I understand the problem that sometimes the scores do not reflect the skills of a particular team.
But I also know that it is incredibly hard to score nuance in team performance. We cannot look at what the robot intends to do or how much it understands its surroundings. (technically we can but this is way too time consuming and subjective). So all we can do is score the end result of their actions. Did they place the item? Did they adress the correct person?
I think the key here is not to try to score things which are hard to observe. But do design the challenges in such a way that the robots need this understanding in order to perform the right actions.
The current scoring system does not accurately reflect how well teams perform. There’s a gap between the numerical scores and the actual competence of the teams, which suggests a need for improvement in the way robots semantically understand and reflect on tasks.
Problem:
Scores often fail to capture the nuances of team performance, leading to potentially misleading representations of team competence. The lack of meaningful semantic understanding in robots contributes to this issue. Robots need to better understand the context and significance of actions to evaluate team performance more accurately.