osrf / vorc

Virtual Ocean Robot Challenge (VORC) resources
Apache License 2.0
38 stars 11 forks source link

add new exit flag to gymkhana channel scoring plugin #35

Closed mabelzhang closed 3 years ago

mabelzhang commented 3 years ago

Depends on osrf/vrx#233

Fixes problem: When boat passes the last set of navigation gates in the gymkhana task, shutdown signal is issued to Gazebo.

A new SDF parameter flag per_plugin_exit_on_completion is added.

After these two PRs are merged, we will need to test vrx-docker with ghostship solution and make sure Gazebo no longer shuts down prematurely.

It's a tiny bit hacky, because now we have two flags for "exiting upon completion." One specified from the environment variables, and another from the SDF parameter. This is because we are chaining multiple scoring plugins for a single task. They are slightly different in that, the env var is for all the plugins in the current shell, and the SDF parameter is just for one specific plugin. The dilemma is how to resolve the two when one is true, the other is false.

Currently, I have the env var overwrite the SDF parameter. Alternatively, we can do an AND or OR, but that might make the vrx-docker evaluation more complicated, because the evaluation run_trial.bash issues VRX_EXIT_ON_COMPLETION=true for all tasks. We currently don't say, issue it on all tasks except gymkhana. In the overwrite behavior, we can set VRX_EXIT_ON_COMPLETION=false, and explicitly specify the SDF parameter flag in each world xacro file. This I think is the cleanest way, though it adds a few lines of SDF.

To test:

Keep this running in a terminal:

rostopic echo /vorc/gymkhana_channel/task/info

For all cases below, once the topic state field changes to running, move the boat by keyboard / mouse / rostopic pub straight forward. Observe what happens when it passes the last set of gates.

Case 1: Original behavior

Comment out the new line <per_plugin_exit_on_completion>false</per_plugin_exit_on_completion>.

roslaunch vorc_gazebo gymkhana.launch 

Expected: When boat passes last gates, gzserver is killed in terminal. gzclient will still be open but no further steps are simulated. No new messages in the rostopic echo above.

Case 2: Added behavior

Uncomment the new line <per_plugin_exit_on_completion>false</per_plugin_exit_on_completion>.

roslaunch vorc_gazebo gymkhana.launch 

Expected: When boat passes last gates, Gazebo continues running. The state field in rostopic echo above changes to finished, score remains the same for all subsequent messages.

Additionally, I tested by echoing the two tasks in two separate terminals

rostopic echo /vorc/gymkhana_blackbox/task/info
rostopic echo /vorc/task/info

and saw that when the top-level task "finished" with 0 second left, sub-task blackbox "running" with 1 second left (due to small differences in transport maybe), shutdown was still issued to gazebo, by design. That verifies that the top-level scoring plugin's time is respected (<per_plugin_exit_on_completion> is false for the two sub-tasks, true for the top-level task.)

Case 3: Original environment variable overwrites added behavior

Uncomment the new line <per_plugin_exit_on_completion>false</per_plugin_exit_on_completion>.

VRX_EXIT_ON_COMPLETION=true roslaunch vorc_gazebo gymkhana.launch 

Expected: Same as case 1.

I hope nothing else is broken, but more thorough tests to find bugs are welcome.

crvogt commented 3 years ago

Hey @mabelzhang, were you able to get this working with vrx-docker? I managed to reproduce the functionality described above, but having the same gate exit results with a fresh clone. My theory was either I was being dense and missed something, or perhaps the task generation isn't generating the files correctly?

mabelzhang commented 3 years ago

Thanks for testing it! Yes, you would need to use the branch in this PR https://github.com/osrf/vrx-docker/pull/25

crvogt commented 3 years ago

Thanks for testing it! Yes, you would need to use the branch in this PR osrf/vrx-docker#25

Ah, density it is! Test/scores indicate to me that it's working. Thank you!