Implementation of node tests to ensure planning node stability under exceptional inputs

kyoichi-sugahara commented 1 year ago

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description

background

In each node of the planning module, when exceptional input, such as unusual routes or significantly deviated ego-position, is given, the node may not be prepared for such input and could crash. As a result, debugging node crashes can be time-consuming. For example, if an empty trajectory is given as input and it was not anticipated during implementation, the node might crash due to the unaddressed exceptional input when changes are merged, during scenario testing or while the system is running on an actual vehicle.

Important Notes

During test execution, when launching a node, parameters are loaded from the parameter file within each package. Therefore, when adding parameters, it is necessary to add the required parameters to the parameter file in the target node package. This is to prevent the node from being unable to launch if there are missing parameters when retrieving them from the parameter file during node launch.

Purpose

The purpose of implementing node test is ensure that node operates correctly when receiving exceptional input.

planning_test_utils package provides a utility for implementing node test. By utilizing this utility and implementing node test for exceptional input, we aim to to reduce bugs that are only discovered when actually running the system, by requiring measures for exceptional input before merging PRs.

Possible approaches

Implement node test by utilizing planning_test_utils package

Node tests are implemented with the file name of test_module_neme_node_interface.cpp

Implemented node tests

Node	Test name	exceptional input	output	Exceptional input pattern
planning_validator	NodeTestWithExceptionTrajectory	trajectory	trajectory	Empty, single point, path with duplicate points
planning_validator	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position
motion_velocity_smoother	NodeTestWithExceptionTrajectory	trajectory	trajectory	Empty, single point, path with duplicate points
obstacle_cruise_planner	NodeTestWithExceptionTrajectory	trajectory	trajectory	Empty, single point, path with duplicate points
obstacle_cruise_planner	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position
obstacle_stop_planner	NodeTestWithExceptionTrajectory	trajectory	trajectory	Empty, single point, path with duplicate points
obstacle_stop_planner	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position
obstacle_velocity_limiter	NodeTestWithExceptionTrajectory	trajectory	trajectory	Empty, single point, path with duplicate points
obstacle_velocity_limiter	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position
obstacle_avoidance_planner	NodeTestWithExceptionTrajectory	trajectory	trajectory	Empty, single point, path with duplicate points
scenario_selector	NodeTestWithExceptionTrajectoryLaneDrivingMode NodeTestWithExceptionTrajectoryParkingMode	trajectory	scenario	Empty, single point, path with duplicate points for scenarios:LANEDRIVING and PARKING
scenario_selector	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position
freespace_planner	NodeTestWithExceptionRoute	route	route	Empty route
freespace_planner	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position
behavior_path_planner	NodeTestWithExceptionRoute	route	route	Empty route
behavior_path_planner	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position
behavior_velocity_planner	NodeTestWithExceptionPathWithLaneID	path_with_lane_id	path	Empty path
behavior_velocity_planner	NodeTestWithOffTrackEgoPose	odometry	odometry	Off-lane ego-position

Unimplemented node tests

In cases where there are variations of modules operating within nodes, such as behavior_path_planner and behavior_velocity_planner, we need to configure which specific module is active during testing. Alternatively, we may need to conduct tests under conditions where all modules are operating.

Implementation of tests that ensure the operation of all modules within the behavior_velocity_planner, and tests configured to activate specific nodes within the behavior_path_planner, has not been done yet.

Moreover, tests for special situations, such as those confirming that nodes remain operational when the vehicle is driven in reverse, or when the vehicle's direction undergoes a significant change, are still unimplemented. The introduction of these tests is essential to assure the system's robustness under a variety of conditions.

Definition of done

The implementation of node tests, following the configuration for the operation of individual scene modules within the behavior_velocity_planner and behavior_path_planner, and the assurance of no node failure are confirmed. If test failure occurs, countermeasures should be applied.

The implementation of tests for other special inputs, as mentioned in 'Unimplemented node tests', is not included in this issue.

tasks

[ ] feature to verify within the test that the target scene module is running for behavior_velovity_planner
[ ] feature to verify within the test that the target scene module is running for behavior_path_planner
[ ] creation of a document to support test implementation

behavior_path_planner

[x] lane_following
[ ] lane_chnage
[ ] avoidance
[ ] side_shift
[ ] dynamic_avoidance
[ ] goal_planner
[ ] start_planner

behavior_velocity_planner

[x] blind_spot
[x] intersection
[x] crosswalk
[x] detection_area
[ ] no_stopping_area
[ ] out_of_lane
[ ] run_out
[ ] speed_bump
[x] traffic_light
[ ] virtual_traffic_light

stale[bot] commented 1 year ago

This pull request has been automatically marked as stale because it has not had recent activity.

stale[bot] commented 11 months ago

This pull request has been automatically marked as stale because it has not had recent activity.

autowarefoundation / autoware.universe