Strengthen collaboration with atHome Education league

[ARTenshi]

Is your idea/suggestion related to a problem? Please describe.

Recently, there have been several discussions on lowering the atHome level and on introducing more partial scoring so any team can score; however, RoboCup at Home Education league does that: it takes the at Home rulebook, selects relevant tasks, and evaluates partial skills. Therefore, instead of focusing our efforts on the lower edge of at Home, we should encourage new teams to gain some experience by participating in the Education league.

Describe the solution you'd like

By having a closer collaboration with the Education league, we can move forward on the domestic service robots higher edge of the at Home league. Some experienced teams have raised their concerns on constant lowering the level of the tasks and on pulverised scoring that might allow robots to score without solving a given task -- e.g. autonomously taking two objects and placing them in their correct place may score less than taking "for transportation" five objects, rotating the robot and throwing them away. It has been argued that "it depends on the team's strategy", however the only options we should allow to teams are on how to solve a task (for example, by using feature or feature-less object recognition or 2D vs 3D navigation) and not whether they want to solve the task or not.

[LeanderVonSeelstrang]

I think we need to push both boundaries, the lower and the upper, outward. We have to strictly avoid designing a task in such a way that you can exploit some aspect of it.

But the analogous problem in the other direction is also problematic: In the previous rulebook, we had some task elements that no rational points-maximizing team will ever tackle.

Score design is a balancing task and presents similar difficulties to curriculum design.

I believe that increasing complexity mainly leads to fewer points, not faster progress. At least not as long as a team does not have more than 90% of the points in all tasks.

But you are absolutely right that points glitches must be identified and avoided at all costs. And that small-step scoring is excellent for introducing just such glitches into the rulebook.

You brought the example of transportation, instead of real interacton. Leroy brought the example of navigation without any interaction.

Both problems could be solved with point group multipliers. Each partial score gets points, but only if all scores of a group are solved the initially neutral multiplier is increased. (Deus-Ex-penalties could also be introduced multiplicatively. Whenever a person helps the group multiplier is halved). Sven has also already noted that it might be interesting to consider the idea of diminishing marginal utility in scoring: The more often the exact same subtask is solved, the less points the solution gives. This would reward diversity of skills (and conversely penalize speed).

Probably it is important to shift the focus of the discussion towards the upper bound. At the same time, it must be said that only Team Tidybot solved the task confidently and they can probably still deal with communication optimization (because grappling and navigation is really solved).

In summary, I don't think it's an either or problem. We can have both at once: easier entry and reward of higher complexity and stability.

And regarding cooperation: Yes, more cooperation is always good, but I don't think that would solve the problem either.

[ARTenshi]

Ok, I agree with all that and that would be a good start.

A redesign of the tasks with weighted factors might help to progress in the right direction -- and I am not suggesting harder tasks but meaningful ones, for example, in Storing Groceries it makes no sense to have a set of objects in a grid configuration (one object per square) on a table next to the shelf (i.e. why did the user order the objects on the table and ask the robot to put them in the shelf instead of ordering directly him/herself?), it would be more meaningful to have the objects inside a (delivery) box and then, maybe, inside the shelf, if there are four categories, having only one object of one category and three other empty spaces and the robot should decide if the current object belongs to this category or should assign an empty region to a new (known) category. The same for Clean the Table, having the objects in a random configuration along with left food, garbage, and some transparent glasses. (These are just examples of adding meaning to the tasks, I am not saying that they should be like that.)

Regarding collaboration, it might not solve the problem but would provide a good baseline to develop the tasks (i.e. not assuming that there might be teams with zero experience).

[hawkina]

We could state in the rules that the robot should aim at finishing the task, and things like pick up objects + throwing them away just to get the pick up points are illegal. Last year it seemed like it happened multiple times that a robot almost completed a task, but failed at the very last section of it, and got no points for it whatsoever. That is a rather frustraiting experience, mostly if the team actually attempts pick and place instead of asking for help. But yes, giving more rewards while avoiding exploitation is difficult.

Strengthtening the bond with the Education League is a good idea, but to my knowledge they are not an official League yet and the Education competition does not always happen, which makes this challenging.