osrf / vrx

Virtual RobotX (VRX) resources.
Apache License 2.0
389 stars 178 forks source link

Adjust practice worlds to make use of new wavefield model envelope #686

Closed M1chaelM closed 1 year ago

M1chaelM commented 1 year ago

This PR adjusts the wavefield parameters for the practice worlds to cover a broader spectrum of possibilities using our recently updated wavefield envelope (see #684). The changes introduced to the worlds are as follows:

To test:

tejalbarnwal commented 1 year ago

Hey, i observed some of the things for difficult worlds(_2 worlds):

  1. For the station-keeping world, in the absence of any vehicle control, the vehicle, after some time, collides with the beach. I have attached a video simulation(please watch it at 2x speed). I think this won't cause an issue when teams start applying controls but I am concerned about the extent of control that would be required to tackle this. A sort of similar situation arises with wayfinding task2 as well, where without control, the robot rests against the beach. video
  2. for the wildlife2 task, the crocodile and turtle do get submerged inside the water periodically. Since teams rely on pose topics to know the position of wildlife, this shouldn't be an issue ig, right?
  3. With acoustic_perception2, acoustic_tracking2 and scan_dock_deliver2, the vehicle drifts along the direction of the waves and wind, which I believe is a natural behavior(coz there is no application of controls). Eg: video
  4. Finally, in case of perception_task2, I observed buoys to bounce up when they are spawned in the environment. I wanted to confirm, if this is expected behaviour?video

I believe that due to the gentle wind speed and moderate wave conditions, the easy (_0 worlds) and medium-difficulty level worlds (_1 worlds) will not experience significant impacts. Should I conduct a tests on them after 5-6 hours?

M1chaelM commented 1 year ago
  • For the station-keeping world, in the absence of any vehicle control, the vehicle, after some time, collides with the beach. I have attached a video simulation(please watch it at 2x speed). I think this won't cause an issue when teams start applying controls but I am concerned about the extent of control that would be required to tackle this. A sort of similar situation arises with wayfinding task2 as well, where without control, the robot rests against the beach. video
  • for the wildlife2 task, the crocodile and turtle do get submerged inside the water periodically. Since teams rely on pose topics to know the position of wildlife, this shouldn't be an issue ig, right?
  • With _acousticperception2, _acoustictracking2 and _scan_dockdeliver2, the vehicle drifts along the direction of the waves and wind, which I believe is a natural behavior(coz there is no application of controls). Eg: video
  • Finally, in case of _perceptiontask2, I observed buoys to bounce up when they are spawned in the environment. I wanted to confirm, if this is expected behaviour?video

Thanks for this very helpful and detailed review. These observations all sound like they are within the range of expected behavior, but I will double check especially the stationkeeping and wayfinding worlds. I agree with your assessment of the easy and medium worlds. I think if you could just visually inspect the code to make sure the parameters are consistent, and maybe try out 1-2 easy and 1-2 medium worlds chosen at random that would be enough.

I'm not sure what you mean by "after 5-6 hours." Do you mean the next time you'll be available to review is in 5-6 hours? If so, that's fine, and thanks again for taking a look.

tejalbarnwal commented 1 year ago

I'm not sure what you mean by "after 5-6 hours." Do you mean the next time you'll be available to review is in 5-6 hours? If so, that's fine, and thanks again for taking a look.

Yup

  1. Hey, the follow_path0 world had a non-zero gain value. I updated its value to 0.0.
  2. I noticed a similar bouncing effect in perceptation_task1 as I did in perception_task2. The bouncing of balls and buoys was also present here, but I found it to be more pronounced in this case. video
M1chaelM commented 1 year ago
  1. For the station-keeping world, in the absence of any vehicle control, the vehicle, after some time, collides with the beach. I have attached a video simulation(please watch it at 2x speed). I think this won't cause an issue when teams start applying controls but I am concerned about the extent of control that would be required to tackle this. A sort of similar situation arises with wayfinding task2 as well, where without control, the robot rests against the beach. video

I watched your videos and also replayed the hard stationkeeping and wayfinding tasks. The behavior you're seeing is consistent with what we've done in previous competitions, so I think it's OK. In fact, I asked the same question the first time I saw the same thing, and the consensus was that it is actually fairly realistic, and not too uncommon in real-world competitions. If a team leaves their platform in the water without control it can float away and sometimes that means it ends up back on the beach.

M1chaelM commented 1 year ago

2. for the wildlife2 task, the crocodile and turtle do get submerged inside the water periodically. Since teams rely on pose topics to know the position of wildlife, this shouldn't be an issue ig, right?

I looked at this more closely and I will pose the question to the group. On the one hand, this effect also seems physically realistic to me (e.g. sometimes crocodiles go under water), but it would also be fine to reduce the gain a little bit. I made a gif to show the effect at the current setting of period 2, gain 2:

wildlife_period2_gain2

Here's what it would look like at period 6, gain 0.8: wildlife_period6_gain0_8

I think this second one looks a little more realistic and is sufficiently challenging, so I will switch to that for now.

M1chaelM commented 1 year ago
  • With _acousticperception2, _acoustictracking2 and _scan_dockdeliver2, the vehicle drifts along the direction of the waves and wind, which I believe is a natural behavior(coz there is no application of controls). Eg: video

Yes, this looks fine.

  • Hey, the _followpath0 world had a non-zero gain value. I updated its value to 0.0.

Thanks!

  • I noticed a similar bouncing effect in _perceptationtask1 as I did in _perceptiontask2. The bouncing of balls and buoys was also present here, but I found it to be more pronounced in this case. video

Yeah, good observation; this does not look good. It is expected that the buoys should drop into the water and then float back up, but they should not bounce off the surface like basketballs, so something is a bit weird here. I don't think this is a problem with this task or the environmental settings, so it shouldn't block this PR. However, it would be worth presenting your second video to the technical team and creating an issue to look into possible problems with our buoyancy plugin.

M1chaelM commented 1 year ago

Let me know if I've missed anything. If the above comments address your concerns, could you please approve this PR?

tejalbarnwal commented 1 year ago

Let me know if I've missed anything. If the above comments address your concerns, could you please approve this PR?

I guess we have covered it all. Sure.