avoidance ignores pedestrian if there is a bus behind

[VRichardJP]

Checklist

• [X] I've read the contribution guidelines.
• [X] I've searched other issues and no duplicate issues were found.
• [X] I'm convinced that this is not my fault but a bug.

Description

In behavior_path_planner module, it is possible to disable avoidance for some obstacles, such as pedestrian.

For example in the following situation, a pedestrian is on the path and the vehicle does not try to avoid it:

However, if there is another obstacle that can be avoided behind the pedestrian, then the vehicle tries to avoid both:

I guess "unavoidable" obstacles have been introduced to avoid risky manoeuver (e.g. the pedestrian will likely not stay on the middle of the road, so it is better to simply wait). In such case, the vehicle should stop in front of the pedestrian like in the first screenshot.

Expected behavior

The vehicle never avoid non-target obstacle.

Actual behavior

non-target obstacle is avoided under some circumstances

Steps to reproduce

see pictures above

Versions

No response

Possible causes

No response

Additional context

No response

[zulfaqar-azmi-t4]

In the second image, it does seem like to non-target obstacle was ignored. But just in case, can you run

ros2 topic echo /planning/scenario_planning/lane_driving/behavior_planning/behavior_path_planner/debug/avoidance_debug_message_array

I think for the pedestrian ObjectIsNotType will be printed in the second

avoidance_info:
- object_id: ''
  allow_avoidance: false
  longitudinal_distance: 0.0
  lateral_distance_from_centerline: 0.0
  to_furthest_linestring_distance: 0.0
  max_shift_length: 0.0
  required_jerk: 0.0
  maximum_jerk: 0.0
  failed_reason: ObjectIsNotType
- object_id: 6cee772d36247e61b13266516b53093b
  allow_avoidance: true
  longitudinal_distance: 24.38448885981013
  lateral_distance_from_centerline: 1.5808671530619405
  to_furthest_linestring_distance: 5.532324530608035
  max_shift_length: 4.684824530608035
  required_jerk: 0.005692831476198248
  maximum_jerk: 1.0
  failed_reason: ''

If it is so, then the pedestrian object was ignored.

[VRichardJP]

Yes, the avoidance_info contains one entry for the pedestrian with ObjectIsNotType. So if I understand correctly it means the avoidance does not try to avoid the pedestrian. In practice however, the pedestrian is avoided.

[shulanbushangshu]

Yes, pedestrians are avoided indirectly.If you consider avoiding dangerous operations,additional rules need to be added.

[zulfaqar-azmi-t4]

Yes, if pedestrian is registered as non avoidable object, the generated avoidance path will be consider them. In your second case, it just happens that the pedestrian is nearby the bus and it "seems" as if it is avoiding the pedestrian.

By default, if it is not avoidance's object, if it is still blocking ego's route, ego will decelerate and stop.

You can get further information by turning on the debug markers.

[VRichardJP]

Yes, I understand how the avoidance logic is implemented.

My point is whether it makes sense for the avoidance to "ignore" some class of obstacles. If this is for the safety of the operation, then it totally makes sense in the 1st scenario, however it is counter-productive in the 2nd scenario.

Indeed, because the pedestrian is ignored by the avoidance algorithm in the 2nd scenario, it can make the ego vehicle drive super close to the pedestrian, which is certainly not safe at all:

[zulfaqar-azmi-t4]

I see. My apologies, before proceeding, let me correct my understanding as follows

1. Based on your report, the current behavior is similar to the Expected Behavior since the vehicle never avoid non-target obstacle, which in this case is the pedestrian.
2. However, your concern is about why the parameter doesn't set pedestrian as avoidable object as default.

Is these correct?

[VRichardJP]

Based on your report, the current behavior is similar to the Expected Behavior since the vehicle never avoid non-target obstacle, which in this case is the pedestrian.

I think I have confused everyone with the word "avoid", because we understand different things:

• The avoidance algorithm does not take into account the pedestrian when it generates the avoidance trajectory. From this point of view, the planner does not avoid the pedestrian
• In practice, the vehicle generates a trajectory that goes around the pedestrian. From this point of view, the planner does avoid the pedestrian. <- this is what I mean.

So when I say the expected behavior is that the vehicle should never avoid non-target obstacles, I mean no trajectory should be able to go around the object, even if it is unintentional.

It is maybe just the way I interpret the target_object parameter, but I think it does not make much sense to have any target_object.xxx=false if the current behavior is correct. In other word, we should either:

• remove the target_object option, and have the avoidance planner avoid everything
• make the car stop in front of the pedestrian in my 2nd scenario.

[zulfaqar-azmi-t4]

I see.

I think we cannot assume that all user always have access to good perception inputs, therefore it make sense to allow the object type to be configurable.

On the other hand, since we also have the run out module in the behavior_velocity_planner that suppose to handle the pedestrian's scenario.

[mehmetdogru]

@VRichardJP @zulfaqar-azmi-t4

I think there are limited things we can do in behavior_path_planner(bpp) when we consider its coverage. Also run_out module in behavior_velocity_planner is not sufficient in this scenario. What it can be done might be an improvement in obstacle_stop_planner. I prepared some videos to explain why I think so:

---

1) In the first case, there are a static pedestrian and a parked bus behind it. In this scenario ego vehicle avoids the parked bus -not the pedestrian- but at the end it drives very closely to the pedestrian anyway.

https://user-images.githubusercontent.com/48479081/199532222-822ae50d-093f-4f6c-ad41-c9479b724594.mp4

This is not what we want so as @xmfcx suggested in ASWG meeting yesterday; in bpp when it is deciding for shift points (in avoidance module) it can be checked if there are any objects on the path -which we don't wanna avoid such as pedestrians- existing closer than the target obstacle. And it would work on this case.

---

2) However if pedestrian appears next to the path after avoidance is done then there is nothing can be done in bpp layer. Because the pedestrian is not on the path. At the end ego vehicle would drive closely to the pedestrian.

https://user-images.githubusercontent.com/48479081/199532909-d130331c-c5b6-4d7f-af4c-ca36eb9e8bbb.mp4

---

3) We can launch run_out module as @zulfaqar-azmi-t4 suggested. However it wouldn't solve the problem. Ego vehicle would drive closely to the pedestrian anyway.

https://user-images.githubusercontent.com/48479081/199533613-f97f1e4f-c787-4244-b33b-b998d0009859.mp4

---

As a solution we could increase lateral_margin parameter in obstacle_stop_planner:

https://github.com/autowarefoundation/autoware.universe/blob/d10310ddbbf5aaba20858173cb26e7fdcb30ea49/launch/tier4_planning_launch/config/scenario_planning/lane_driving/motion_planning/obstacle_stop_planner/obstacle_stop_planner.param.yaml#L17

However it would be valid for all the obstacle points and ego vehicle would stop for the cases we don't want it to stop (narrow roads, close vehicles on the opposite/neighbor lane). So what we can do is that in obstacle_stop_planner for the points belong to the selected objects (pedestrians in this case) lateral_margin can be larger. I believe it would solve this problem.

What are your opinions on this?

[beyzanurkaya]

I want to work on this approach. I will create an issue and PR about it.

[VRichardJP]

@mehmetdogru Eventually, I think the planner should be able to reassess the situation and invalidate an avoidance path if it does not make sense anymore. For example, if the avoidance path is now blocked half-way, the vehicle should stay on its lane (or go back to it), and wait until the path is clear.

In your second video, if a pedestrian appears on the path (or if lateral_margin is high enough), the vehicle will stop in front of it and wait like this in the middle of the road. It is definitely better than driving 10cm away from the pedestrian, but it is not an ideal situation either.

[mehmetdogru]

@VRichardJP

I agree. Maybe avoidance path can be reverted in such cases.

Although, I am afraid just checking around the path will not be enough considering even though path is reverted, ego vehicle sometimes won't have time to go back to it and stay in the middle of the road anyways. So since run_out module is supposed to handle pedestian scenarios as @zulfaqar-azmi-t4 said, maybe something like run_out module can be implemented in bpp which considers the predicted paths of target objects or in such scenarios bvp can send a message back to bpp so the avoidance path can be reverted on time.

[mehmetdogru]

@satoshi-ota

Could you kindly provide your ideas about the issue. I learnt that you are working on avoidance module for further improvements. I would really like to hear your opinion about this as well.

[satoshi-ota]

@VRichardJP @mehmetdogru @beyzanurkaya @shulanbushangshu @zulfaqar-azmi-t4

Thanks for the discussion. Let me check one thing.

You do not want the vehicle to drive near the pedestrian only when the vehicle is in avoidance maneuver, right ? Or, you don't want not only when the vehicle is in avoidance maneuver but also when the vehicle is driving lanes without avoidance.

If it is the former, then we have the following options:

1. Add a special obstacle stop function to the avoidance function as @mehmetdogru said here For example, if there are obstacle between the avoidance target and the Ego, the Ego stops on the spot.
2. Implement features to avoid pedestrians

In the latter case, then we have the following options:

1. Tuning parameters of obstacle_stop_planner or improve obstacle_stop_planner feature as @mehmetdogru said here
2. Use obstacle_cruise_planner.The behavior of obstacle_cruise_planner is basically the same as the function of obstacle_stop_planner with respect to obstacle stopping, but by default it determines if the Ego should stop based on not pointcloud but object recognition.

In the current architecture, the feature that generates avoidance paths is separated from the feature that stops for obstacles, and I do not want to merge those features, so I think it is better to fix this issue by improving the obstacle stop functions (obstacle_stop_planner, obstacle_cruise_planner, run_out). cc @TakaHoribe

[VRichardJP]

@satoshi-ota I think the vehicle behavior should be consistent over the several scenarios. For example:

• if during normal lane driving we must keep a lateral safety distance of 1.0 meter, we must keep the same safety distance of 1.0 meter during avoidance (I think it is what obstacle_stop'planner already does)
• if the planner is not allowed to avoid pedestrian, it should not matter what is behind the pedestrian (this ticket issue)
• generally, I think avoidance manoeuver should work as a whole: either we can avoid everything and we go, or we can't do he manoeuver and we wait. I think it is not the case yet: autoware always goes as far as it can, even if it means stopping in the middle of the road
• the consequence of the previous point is that if a non avoidable appears halfway our manoeuver, we should avoid it to finish our manoeuver. In other words, the avoidable/non-avoidable does only make sense when autoware has to decide whether to start avoidance or not. Once you are manoeuvering, I think it is best to finish the manoeuver as soon as possible (even if it means avoiding "non-avoidable" objects)
• another consequence is that an obstacle should not become a "launch ramp" to avoid another obstacle. For instance, if a vehicle is parked on the side of the road (=can be avoided), and right behind we see another vehicle in the middle of the lane (which should not be avoided), then autoware should avoid none (I am not sure whether autoware does this or not)
• it should be resilient to detection noise. Unless detection suddenly improves a lot, the sad truth is that "small" objects (pedestrian, bikes, cycles...) are not recognized by autoware from very far, and late classification is definitely a thing. So it is best if autoware's behavior does not change radically when the obstacle is just a few points, then an "unknown" obstacle, then a "pedestrian", then disappear because it is merged into another "car" obstacle, and so on.

Sorry it's just a bunch of disorganized ideas, but I guess it gives a good idea of the behavior I would expect from autoware in the avoidance situation. I think it shouldn't be too difficult to add the few extra rules above to make the avoidance more predictable/human-like.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity.

[xmfcx]

https://github.com/autowarefoundation/autoware/discussions/3097 Needs to be discussed after the planning arch is updated again.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity.