Shared autonomy challenge added to technical challenges

[jpha226]

Created first draft of the rules for the new technical challenge involving human control of one NAO on a team of two robots.

[arnemoos]

I think a major point that is missing is, that there are no restrictions given what a human operated robot is NOT allowed to do. In the current rules you can simply let another autonomous robot play (with the human player restrictions). Also just a dead man's switch that the human controlled robot is only active when there is some human pressing a button is also not enough to count as a human controlled robot.

In my opinion the human controlled robot should not be allowed to select positions where it should walk autonomously (autonomous approaching of the ball in the near vicinity should be allowed), these decisions should come from the human operator! Also I would force a full coderelease of the remote control software ;)

[jpha226]

Thanks Arne for the feedback here! If I understand correctly, the point is that you think there should be limitations on how much autonomy can be included on the human-controlled robot. So my thinking was to put less restrictions on how much autonomy could be used under the assumption that no team has developed fully autonomous behaviors that would match a human-controlled robot even if the human was controlling via simple walk engine directions. That is, as long as the human doesn't have to control joint angles, my assumption is that more human judgement will lead to better behavior. If my assumption is wrong then I think it is ok because it shows we're making progress towards the 2050 goal.

Out of curiosity, I think you mentioned that NAO devils has remote controlled NAOs for testing. Have you all played against them? It would be useful to know how they perform against your team's code.

Noted about the full release -- good point!

[rvilling]

@jpha226 - I think you could find some form of words to state that the robot must not initiate any high level actions itself without the human operator requesting it. However I agree with your basic thinking and it is important that the human operator is not constrained to remote control every step and can set high level goals such as walk to a pose on the field (avoiding obstacles along the path if needed) or even go to the ball and kick it.

I'm not sure how much we can learn from anyone who has remote controlled NAOs to date unless they also worked with restricted observation constraint you mentioned. If they have the omniscient view that people normally have then the challenge for humans is less. If they require low level control (e.g. constant adjustments to walk speed and direction) then the challenge for the human is much much higher.

[ben47955]

I have kind of mixed feeling from this discussion. For one think it would be hard to come with wording limiting too high level command that it's not too hard to validate. Maybe if we forbid screen like worldstate view it may force teams to work at a lower level.

On the other side, one point of this challenge was to assume the best behavior from the human controller. Giving full freedom would make sense from this perspective.

Another point, it may be interesting to lower the number of packet to have proportional ratio as time and robot count are significantly lower for robot to robot.

[arnemoos]

Out of curiosity, I think you mentioned that NAO devils has remote controlled NAOs for testing. Have you all played against them? It would be useful to know how they perform against your team's code.

We have a joystick controller for the NAOs and tested it with only the video feed. But especially the small correction movements in the vicinity of the ball were hard to do so. Thats why we also thought about a higher level interface.

My general concern I currently see is, that an easy way to compete in this challenge would be to just use two autonomous robots where in defense you set static roles and in offensive you limit the one robot to be not able to shoot on the goal. So for me there is missing some minimum requirements to be qualified to count as a human operated robot?

[rvilling]

@ben47955

I have kind of mixed feeling from this discussion. For one think it would be hard to come with wording limiting too high level command that it's not too hard to validate. Maybe if we forbid screen like worldstate view it may force teams to work at a lower level.

Perhaps there was a misunderstanding. Operating at a high level is what the human operator almost certainly should do - we don't want to force teams to work at a low level. The situation we want to prevent (as identified by Arne) is that the "human operated" robot makes the highest level decisions autonomously without being requested to do so by an operator.

In general there are many ways to cheat in RoboCup. Because it is basically impossible to police this, our policy has almost always been to trust and rely on the honour of teams to stick by the rules. Therefore if the rules can state that the second robot must not be fully autonomous and must not make high level decisions by itself (examples can be given), I would be satisified that teams would try to comply with this. Particularly if coupled with a code release.

[jpha226]

Thanks all for the great discussion! I have edited the rules and pushed.

In my edit I was lax and only said that we require "some form of input from the human" as opposed to "some form of control from the human." My goal is to permit teams to be creative in how they use the human to augment the robot. So one possible technique might be to let the robot be autonomous but use the operator for some improved sensing (e.g., the operator just sends the robot information to help it localize). What would you all think about permitting the operator to provide this kind of helpful information but not actually selecting controls?

[rvilling]

I think the high level goal here is not to see how a human can assist a robot to do its task, but how a human can collaborate with other autonomous robots in a game situation. (I think you've said something like this yourself in the challenge goal section.) To that end, I wouldn't be in favour of allowing the situation where the human operator is just augmenting the robot perception but the robot continues to make all high level decisions itself.

As you've identified yourself, the text saying "some sort of input from the human controller" needs to be tightened up a bit.

On a more detailed note:

• Do we need to say that human operators must also not be looking at any live feed from the game (e.g. youtube)?
• The terminology varies in the text between human-operated and human-controlled at least and then we also talk about operating and controlling the robot. It is probably best to pick one term and stick to it.
• I think you should use the word "goal keeper" rather than any shorter version in the text

[ben47955]

I think we may need more tiebreaker. I would suggest something like : 1) Overall points 2) Best average 3) Higher score 4) Flip coin

Other point, state are usually represented in the rule with \texttt{set}. There are usually no capitals.

[ben47955]

Should be clarified if defending teams can make points by passing.

[ben47955]

About yesterday discussion about tie-breaker, I have think about adding goals count and pass count

1)Overall points 2)Higher score 3)Pass counts 4)Goals counts 5)Flip coin