Planning module's output when having PLANNING_ERROR

[YuqiHuai]

Description of Scenario I have generated a scenario (record file attached below) with Apollo 7.0 using SimControl and noticed an abnormal planning output. The scenario's cyber record, full planning output, and a script to extract those planning messages are all included in the attachment.

Consider the following planning message (only key information included, full file in attachment), vehicle was traveling at about 3 m/s before this message was generated:

header {
  timestamp_sec: 1660774996.0802026
  module_name: "planning"
  sequence_num: 298
  lidar_timestamp: 0
  camera_timestamp: 0
  radar_timestamp: 0
  status {
    error_code: PLANNING_ERROR
    msg: "Failed to init reference line info."
  }
}
debug {
  planning_data {
    adc_position {
      header {
        timestamp_sec: 1660774996.034947
        module_name: "SimControl"
        sequence_num: 2191
      }
      pose {
        position {
          x: 587044.6321165494
          y: 4141582.055045245
          z: 0.0
        }
...
        linear_velocity {
          x: 3.143305029907031
          y: -0.05644375319608064
          z: 0.0
        }
        linear_acceleration {
          x: -3.3543739905028147
          y: 0.06023387989580283
          z: 0.0
        }
...
}
gear: GEAR_DRIVE
trajectory_point {
  path_point {
    x: 587044.6321165494
    y: 4141582.055045245
    theta: -0.017954887330706715
    s: 0.0
  }
  v: 0.0
  a: 0.0
  relative_time: -0.002025127410888672
}
trajectory_point {
  path_point {
    x: 587044.6321165494
    y: 4141582.055045245
    theta: -0.017954887330706715
    s: 0.0
  }
  v: 0.0
  a: 0.0
  relative_time: 0.09797487258911133
}
trajectory_point {
  path_point {
    x: 587044.6321165494
    y: 4141582.055045245
    theta: -0.017954887330706715
    s: 0.0
  }
  v: 0.0
  a: 0.0
  relative_time: 0.19797487258911134
}
...
decision {
  main_decision {
    not_ready {
      reason: "PLANNING_ERROR: Failed to init reference line info."
    }
  }
}

The current velocity of the ego vehicle is (x: 3.143305029907031, y: -0.05644375319608064), so about 3 m/s; but the planning module's output trajectory points all have v:0.0, a:0.0, and path_point at exactly where the ego vehicle is currently at. Since I am using SimControl, it received this planning message and published the ego vehicle's next localization at this coordinate with speed = 0.0 m/s. In other words, the ego vehicle decelerated from 3 m/s to 0 m/s in less than 0.01 seconds.

My understanding is that, since there is a PLANNING_ERROR, the planer is asking the vehicle to stop immediately (v=0), as opposed to decelerate immediately (a < 0) until stopped (v=0).

Attachments report.zip

Questions

1. When a PLANNING_ERROR happens, is it expected for all trajectory points to have v=0 and a=0?
2. Knowing that the vehicle is already traveling at 3m/s, is it reasonable to produce trajectory_points that all have v=0m/s? There is no way to decelerate that fast.
3. The original scenario was executed by SimControl, which is not the same as a vehicle's actual Control module. When the vehicle's control module receives such planning message (current v = 3 m/s, trajectory_point = 0 m/s), what would it do?

[ntutangyun]

Hi yuqi,

When there is a PLANNING_ERROR due to the fact that it failed to initialize the reference line, apparently it cannot move forward under such scenario. so I think it makes sense to not to plan any more trajectories and let the control module to brake as fast as possible. I'm not entirely sure on this but you may check the source code for the planning module.

I think when the control module receives all zero velocity trajectories, it will decelerate as fast as it can. You may be able to confirm this by manually publishing such trajectory points when you're running with the simulator together.

Since the sim control does perfect control and it does not simulates vehicle dynamics. so it will definitely jumps from 3m/s to 0m/s because the planned trajectory said so.

[YuqiHuai]

Hi @ntutangyun ,

Thanks for your response! So in the scenario, the planning message after the one I included above asks the ego vehicle to accelerate from 0 m/s to 3 m/s instantly. Putting this all together would mean that the vehicle is able to travel at its original speed but for some reason is asked to decelerate to 0 m/s and then accelerate back to 3 m/s.

Dreamview + SimControl seems to be the only "built-in" scenario testing method, and the perfect control relies on the Planning module to produce a physically feasible trajectory. Therefore the question is, when a planning error happens, should the planning trajectory be something like [ v=3, v=2.6, v=2.2, v=1.8, ... v=0 ] which represents a feasible fast decelerating trajecotry; or should it be [ v=0, v=0, v=0, v=0, ... v=0 ], which is a physically impossible trajectory that can only be interpreted by the Control module to hopefully do the right thing? In other words, "exit gracefully" rather than simply not plan at all.

I want to argue it might be safer to produce a controlled decelerating trajectory without changing the heading, rather than an all-0 trajectory. If the Control module interprets all-0 trajectory as "brake as hard as possible", would this cause some extreme high deceleration which in turn causes passenger comfort violation? in more extreme cases, would this over-deceleration cause vehicle skid?

p.s. I'll try to see if I can produce Planning messages and observe the Control module's behavior using Apollo + LGSVL

[ntutangyun]

I think it's ok for the moment for two reasons:

1. if the control module is involved and the vehicle dynamics are correct, then even if the planned trajectory has an abrupt jump in speed for whatever reason. for example from 0m/s to 3m/s, then the physical vehicle itself won't be able to change its speed instantly. assume the frequency is 10Hz. then after 100 ms, say the vehicle is only able to accelerate to 0.5m/s from 0m/s, then in the next planning phase, the planning module will take the latest speed into consideration and fix itself, i.e., planning the speed as 0.5m/s, 0.6m/s, ... 2.5m/s, 3.0m/s, etc.

2. I agree with you that the planning module should implement sufficient fail-safe and fail-operational mechanisms, for example, when a planning error is encountered, a safe decelerating trajectory should be produced. Again, Apollo open-source version is not perfect, and for the current implementation, it depends on the control module to handle the slow deceleration tasks. This is actually also helpful in certain cases.

Note that if the planning module hangs or crashes suddenly, the control module will try to follow its last received trajectory. As a result, if you think about it, it is actually safer for the planning module to produce a fallback trajectory with all 0m/s velocities, such that when the planning module fails to work, the control module will at least be able to follow the fallback trajectory and put the vehicle to an immediate stop. On the other hand, if the planning module tried to produce a deceleration trajectory with the last velocity greater than zero and it crashes. then it's gonna be very dangerous for the control module to keep driving the vehicle, isn't it?

In summary, Apollo open-source is not perfect, and it has not implemented sufficient fail-safe and fail-operational mechanisms. So if you're targeting its fail-safe and fail-operational mechanisms, for sure, you will discover much more issues. So, based on the current implementation and how the planning and control work together, I don't think it's an issue to produce all zero velocity trajectories when it encounters extreme errors such as this one: "reference line failed to initiate", where the planning module usually needs to restart to resolve this issue.

[YuqiHuai]

Your comments are so insightful, it is great to be able to discuss this with you. I got 2 high-level conclusions for now:

1) It can be good if the planning module can be self-sufficient in terms of fail-safe. Since an error is produced and the module will potentially crash, even if a fail-safe fallback trajectory is to be produced, it must reach v = 0 at the end.

2) Certain outputs will seem to be an error of a specific module, but the system as a whole can tolerate such errors.

I agree that Apollo at the moment is not perfect yet. I focus on using SimControl to test only the prediction and planning module. So far this is the only insufficient fail-safe condition I discovered because I noticed the ego vehicle under SimControl achieved a physically infeasible action.

[ntutangyun]

btw I learned the terms fail-safe and fail-operational in the whitepaper: A Framework for September 2018 Automated Driving System Testable Cases and Scenarios

you can check it out although it's a bit old..

[YuqiHuai]

@daohu527 We did some further investigation on this problem and found the following comment https://github.com/ApolloAuto/apollo/blob/aa0c5eb66189b86a724206305712cfb337c07619/modules/planning/on_lane_planning.cc#L185-L186

Which generated trajectory_points at ego's current location all with v = 0.0, as discussed in this post. Intuitively, planning is asking the ego vehicle to freeze at the current location immediately.

Since the comment suggests this will cause unexpected behavior from the controller, I want to try to fix this function by changing the output to a hard braking (a = -4 m/s^2) trajectory following the vehicle's current heading, with velocity such as [ v=3, v=2.6, v=2.2, v=1.8, ... v=0 ]. Do you have any suggestions or thoughts?

[qwetqwe]

It usually has no path to call this fallback function(just for safe). if there is a path，we will use GenerateFallbackSpeed func to generate a fallback speed

[YuqiHuai]

@qwetqwe Does "has no path to call this fallback" mean, if there is no path then this function will be called?

During the scenarios that I was running with Apollo + SimControl, GenerateStopTrajectory occasionally gets called while the ego is traveling with positive speed. Since I am using SimControl, it perfectly executes the decision, so sometimes Apollo travelling at 3 m/s suddenly does a complete stop (0 m/s) and then resumes moving forward at the original speed (3 m/s). This behavior causes the scenario to be unrealistic because Apollo reaches a complete stop without any deceleration. Since GenerateStopTrajectory produces a physically impossible decision, and SimControl does not take that into account. What would you suggest I do to make the scenario realistic?