search

osrf / vorc

Virtual Ocean Robot Challenge (VORC) resources
Apache License 2.0
38 stars 11 forks source link

Odd Behavior in Gazebo #36

Open valschmidt opened 3 years ago

valschmidt commented 3 years ago

I am loath to write this because I am the lone person on our team suffering from it. But in a somewhat desperate attempt, I'm hoping others might at least have some suggestions.

Frequently, (but not all the time), always after beginning to send thrust commands, something crashes/resets (I'm not sure what), and my vessel teleports to the datum location and becomes unresponsive. There are no messages of any kind, and gzserver is still running. I have verified that the reported location is coming from the simulated GPS and not any down-stream localization.

I am running cora in a VMWare virtual machine, without an Nvidia GPU. Nonetheless, my VM is allocated 8 cores and 16 GB and is taxed but does not appear to be swamped in compute or memory. That said, it does not quite run in real-time.

A reboot of the virtual machine has no positive effect.

I do understand enough about Gazebo to know if a plugin can fail without any indication. Nor am I sure how to troubleshoot the problem. I welcome any thoughts.

-Val

caguero commented 3 years ago

Just out of curiosity, are you sending commands by any chance while the task state is in initial? If that's the case, the boat is locked to the world. It's released in the ready state.

valschmidt commented 3 years ago

My immediate work has been in the basic marina model. It didn’t occur to me that there might be task level stuff interfering in that simulation. Is that something I should be monitoring?

On Dec 4, 2020, at 4:54 PM, Carlos Agüero notifications@github.com wrote:

Just out of curiosity, are you sending commands by any chance while the task state is in initial? If that's the case, the boat is locked to the world. It's released in the ready state.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/osrf/vorc/issues/36#issuecomment-739041558, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASPJVZVVF67E32USLGAHA3STFK77ANCNFSM4UN4BUJQ.

Val Schmidt Center for Coastal and Ocean Mapping / Joint Hydrographic Center University of New Hampshire Chase Ocean Engineering Lab 24 Colovos Road Durham, NH 03824 e: vschmidt [AT] ccom.unh.edu m: 614.286.3726

caguero commented 3 years ago

If you're running the marina.launch, then, no scoring plugin should be loaded and the boat shouldn't be locked to the world at any time.

You can try running the simulation with:

roslaunch vorc_gazebo marina.launch --extra_gazebo_args:="--verbose"

to see if you get any errors from Gazebo.

valschmidt commented 3 years ago

We were able to reproduce the failure on another system and tracked it down to my thruster commands. I admit I don't fully understand the what's happening, but if one inadvertently sends very large thrust commands it can cause the effect we've observed. It's surprising to me because I could find code for the thruster model that limits the input commands to (nominally +/- 1). This is no longer a critical item, as we can follow the directions and send commands within +/- 1, and not suffer the problem. I'll leave this issue to others to close in case someone wants to have a look first to see if they can make it more robust.

Thanks, Val

mabelzhang commented 3 years ago

I haven't tried to reproduce this. Just jotting down some notes for reference.

I think these are the lines referred to above, about bounding the commands to [-1, 1] https://github.com/osrf/vrx/blob/8dff85cf7ecfcc3a8b7f98494c732f37e6344865/usv_gazebo_plugins/src/usv_gazebo_thrust_plugin.cc#L309-L332

where maxCmd is by default 1.0: https://github.com/osrf/vrx/blob/8dff85cf7ecfcc3a8b7f98494c732f37e6344865/usv_gazebo_plugins/src/usv_gazebo_thrust_plugin.cc#L231 and it is set to 1.0 in cora_description default SDF as well https://github.com/osrf/vorc/blob/61ef595c96667e14f6f4635e2b1081633069b14f/cora_description/urdf/cora_thruster_config.xacro#L15

One possibility is that in the first snippet above, it's calling UsvThrust::Glf(), passing in some very specific constants. I don't understand the equation and don't know if these constants are specific to the WAM-V and need to be changed for CoRa. If they do, I don't know what they need to be changed to. So that hits the limit of my knowledge :sweat_smile:

To test this without that knowledge, whether that's me or someone else, it might be worth printing in GlfThrustCmd() and making sure the return value is indeed bound to [-1, 1].