search

AIasd / ADFuzz

An open-source software package for fuzzing autonomous driving systems in high-fidelity simulators
MIT License
30 stars 9 forks source link

Reproducing result for the archived paper #15

Closed YuqiHuai closed 2 years ago

YuqiHuai commented 2 years ago

Hi! Thank you for all the responses to earlier issues. I am able to successfully run ADFuzz without any issue. I am trying to run ADFuzz in the same setup as described in the paper at https://arxiv.org/pdf/2109.06126.pdf, 140 simulations, and 100 warm-up simulations. Are the following the command arguments to use? Can I use the default value for all other arguments?

-a nsga2-un --rank_mode adv_nn --warm_up_len 100

AIasd commented 2 years ago

--pop_size is set to 10 for this experiment. --has_run_num should be set to 140 accordingly. It has this warm-up/seed collection stage for which nsga2-un should be run. The folder consists of the seeds is then used as the value for --warm_up_path.

Another possible source of difference is that i used Apollo 6.0 (with Signal Sensor) so if modular testing is used, the results should be different.

Also, since I made multiple changes to the current repository after the results for that paper, if you see very different result, please let me and I will look into possible causes. Some reasonable randomness should be normal since the simulation is not deterministic and the fuzzing process also has some randomness. However, the superiority of the proposed method over the other two baselines should hold.

YuqiHuai commented 2 years ago

Thanks for the explanation!