The rulebook should mandate that we actually solve the technical challenges

[PetervDooren]

Is your idea/suggestion related to a problem? Please describe.

Somewhat related to #805. Many times we see teams make extensive use of prior knowledge to hard-code a solution that gives points at a tournament but which does not solve the technical challenge.

(Issue outlined in TC meeting 31/07/2023)

[PetervDooren]

Easiest solution to solve many cases. Move the furniture slightly between challenges. or otherwise limit teams ability to know exactly what the arena will look like. (this does become harder for heavier furniture such as dishwashers and cabinets.)

[akinobu1998]

Ideas for manipulation tasks:

• Use unknown objects to prevent the robot from bypassing manipulation without perception
• Change furniture location before the task or use the random picking-up location for each test.
• Introduce non-expert helper (volunteer)

Currently, some robots just say the object name to bypass the manipulation, "Give me the xxx." for example. Especially for clean the table, by using some unknown objects as well as the known objects, at least we can prevent this behavior. For example, in storing groceries, store "unknown food" in to "food" or "unknown" category. It is not necessary to know the object name to manipulate objects.

As @PetervDooren mentioned, moving furniture should work, or use random pick-up locations each trial (announce some possible pick-up locations 2 hours before the test).

Introducing a non-expert helper can also make the deus ex machina harder. Non-expert helpers might not know how the robot grasps the object or even the object's name. The robot should clearly state the object that the robot wants to manipulate.

[PetervDooren]

Your idea of using unknown objects would work by having the manipulation objects be a subset of a larger set, so the robot does not know a priori what needs to happen. However in clean the table we can achieve this by using a larger set of cutlery. like the bowl and a mug or cup. The rulebook already specifies that only two of spoon, fork, knife are present.

I would not make deus ex machina harder. The robot is already asking for help, indicating it cant solve the problem. With the scoring strategy described in #808 teams are heavily disincentivised from using deus ex machina. So I think that is already fair.

[ARTenshi]

I wouldn't make deus ex machina harder, either, but I agree with @akinobu1998 on making deus ex machina real (not just open loop voice requests).

[akinobu1998]

For objects in clean the table, in 2023, only cutlery from YCB objects was used. To evaluate the robustness of manipulation, we can also use another type of cutlery (and clean the table is a stage II task focused on manipulation). And, yes, it is already specified in the rulebook, we can achieve it in the object selection phase.

Thank you for your opinion, I would like to propose the idea of a non-expert helper to make the deus ex machina more natural no to make it hard.

[PetervDooren]

This would not prevent hardcoding the task. Teams can still ask for handovers and score big.