ScenarioGenerator

ScenarioGenerator performs search-based software testing of Baidu Apollo by automatic generation of various traffic scenarios in SVL end-to-end autonomous vehicle. to verify and validate the safety functionality of Apollo autonomous systems.

If the code helps your research please cite our paper ("Efficient and Effective Generation of Test Cases for Pedestrian Detection – Search-based Software Testing of Baidu Apollo in SVL" ) presented at AITest 2021 : The IEEE Third International Conference On Artificial Intelligence Testing in the Autonomous Driving AI Test Challenge track.

Find more information about the challenge in this presentation and in this directory.

Scenario Creation using SVL editor: This video demonstrates how simplePedestrian.json scenario is created.
ScenarioGenerator: This video demonstrates ScenarioGenerator.py in action.
Safety violation/failures: This video presents the safety violations that are detected.

Credits

If you use ScenarioGenerator or got inspired by our paper (Efficient and Effective Generation of Test Cases for Pedestrian Detection – Search-based Software Testing of Baidu Apollo in SVL), please link to this project and cite our paper.

@inproceedings{scenariogenerator,
  title = {Efficient and Effective Generation of Test Cases for Pedestrian Detection – Search-based Software Testing of Baidu Apollo in SVL},
  author = {Hamid Ebadi and Mahshid Helali Moghadam and Markus Borg and Gregory Gay and Afonso Fontes and Kasper Socha},
  booktitle = {IEEE AITest 2021},
  year = {2021}
}

Setup

Apollo:

$ git branch r6.0.0
$ bash docker/scripts/dev_start.sh
$ bash docker/scripts/dev_into.sh
## For DEV ONLY (implemented in restart_apollo() function):
# in-dev-docker:/apollo]$ bootstrap_lgsvl.sh stop && bootstrap_lgsvl.sh 
# in-dev-docker:/apollo]$ bridge.sh

Activate the following modules in dreamview: planning - perception transformation routing, localization, control

SVL simulator:

First, configure the sensors as explained in 1, 2. Start the SVL simulator in API-only mode.

Scenario Generator:

Install SVL python API and then you can either edit and use start.sh to use the default settings or manually specify the command line arguments for ScenarioGenerator:

usage: ScenarioGenerator.py [-h] -i INPUT --action ACTION
                            [--vector VECTOR [VECTOR ...]]
                            [--des-forward-right DES_FORWARD_RIGHT DES_FORWARD_RIGHT]
                            [--seed SEED] [--steps STEPS]
                            [--pos-noise-range-xz POS_NOISE_RANGE_XZ POS_NOISE_RANGE_XZ]
                            [--color-noise-range-rgb COLOR_NOISE_RANGE_RGB COLOR_NOISE_RANGE_RGB COLOR_NOISE_RANGE_RGB]
                            [--weather-noise-range WEATHER_NOISE_RANGE WEATHER_NOISE_RANGE WEATHER_NOISE_RANGE WEATHER_NOISE_RANGE WEATHER_NOISE_RANGE]
                            [--time-max-noise TIME_MAX_NOISE]
                            [--speed-max-noise SPEED_MAX_NOISE]

For each scenario, a new JSON file is created in the results directory. Take a look at replay.sh to get an idea of how to replay these JSON scenario files.

Solution for known issues

cleanup.sh script

Remove files and folders in the following directories:

results folder
SVL test results
Apollo core files
.config/unity3d/LGElectronics/SVLSimulator/Analysis & videos

Funding

This work is done by Infotiv AB under VALU3S project in a collaboration with RISE under ITEA3 European IVVES project. This project has received funding from the ECSEL Joint Undertaking (JU) under grant agreement No 876852. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Austria, Czech Republic, Germany, Ireland, Italy, Portugal, Spain, Sweden, Turkey.

The ECSEL JU and the European Commission are not responsible for the content on this website or any use that may be made of the information it contains.