Proposal for general guidelines

[LeanderVonSeelstrang]

Is your idea/suggestion related to a problem? Please describe.

You wanted to develop a set of guidelines to design the rulebook. Here is what I think is desirable.

Describe the solution you'd like

• Push the field forward to actually useful household robots
• Measure the progress of the field in a meaningful way (become a benchmark)
• A general upward tendency should be visible over the years (at least if there is progress; currently no trend is visible because no one is compiling statistics, previously a downward trend was visible due to increasing difficulty. You don't want to be in the situation of having to justify a downward trend).
• Strive for a fair scoring process:
• Reward difficult or innovative solutions
• Track the success rate of atomic tasks
• Be comparable within a tournament (intra fairness)
• Be comparable across years (inter fairness)
• Maintaining progress already made
• Low entry threshold to recognize new teams' accomplishments as well, increasing their motivation (and possibly funding) for future years (don't be stingy with points)
• Provide and reward the use of arena interfaces, as this will attract a larger audience (thus increasing the relevance of the competition)
• Be aware that anything not explicitly rewarded with points is likely to be ignored (arena interfaces such as VizBox, robot view, and robot audio output streaming)
• Don’t punish features you desire that require work to implement (e.g. pointing at objects as HRI. No team will implement that as it is harder than just recognizing the object). Just design the tasks in a way that you can reward those features if you want them implemented.
• Minimize rulebook changes over the years (especially in the entry stages)
• Minimize the size for the rulebook
• Minimize the workload for the referees and volunteers
• Maximize the relevance of the content of the rulebook
• Be clear, not polite
• Be precise and avoid ambiguity
• The rulebook should be as unpolitically as possible
• The rulebook should be graphically pleasing (Style, Layout, Icons, Readability). Always pick functionality of style though
• Don't call the "demonstration" a "test", call it a „performance" or „demonstration“.
• Best case scenario: people who are not in the robotics field should still hear about this contest. Ask yourself how you can reach a broad audience.

Additional context

The guidelines should not end up in the actual rulebook.

[PetervDooren]

I fear these guidelines are not usefull due to vagueness in terms such as 'fair', 'the field'. They are more values that we want to strive for. They must be made more concrete to become requirements and then we need to know HOW to meet these requirements. And this entire process will probably not go without tradeoffs.