Running ADFuzz for more than 1 hour

[YuqiHuai]

Hi @AIasd , hope you are doing well!

Problem Summary

I am trying to run ADFuzz with the following arguments

python3.8 ga_fuzzing.py --simulator svl \
    --n_gen 60 --pop_size 10 --algorithm_name nsga2-un --rank_mode adv_nn \
    --has_run_num 140 --objective_weights -1 1 1 0 0 0 0 0 0 0 \
    --check_unique_coeff 0 0.1 0.5 --episode_max_time 30 \
    --ego_car_model apollo_6 --route_type 'BorregasAve_left' \
    --scenario_type 'turn_left_one_ped_and_one_vehicle' \
    --record_every_n_step 5 --n_offsprings 50

• pop_size and has_run_num is configured based on https://github.com/AIasd/ADFuzz/issues/15#issuecomment-1079088700

• some other values are configured based on https://github.com/AIasd/ADFuzz/blob/main/doc/stack2_svl_apollo.md

• --n_gen is set to 60 because I want to generate about 600 scenarios.

However, after 100 scenarios, main will exit with the following error message

  File "/home/yuqi/ResearchWorkspace/Reproduction/ADFuzz/pymoo/pymoo/model/algorithm.py", line 291, in _solve
    self.next()
  File "/home/yuqi/ResearchWorkspace/Reproduction/ADFuzz/pymoo/pymoo/model/algorithm.py", line 263, in next
    self._next()
  File "ga_fuzzing.py", line 920, in _next
    self.set_off()
  File "ga_fuzzing.py", line 822, in set_off
    clf = train_net(X_train, y_train, [], [], batch_train=200, device_name=self.device_name)
  File "/home/yuqi/ResearchWorkspace/Reproduction/ADFuzz/fuzzing_utils/pgd_attack.py", line 742, in train_net
    for i, (x_batch, y_batch) in enumerate(train_loader):
  File "/home/yuqi/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 628, in __next__
    data = self._next_data()
  File "/home/yuqi/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 671, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/yuqi/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/yuqi/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/yuqi/ResearchWorkspace/Reproduction/ADFuzz/fuzzing_utils/pgd_attack.py", line 36, in __getitem__
    return (self.X[idx], self.y[idx])
IndexError: index 94 is out of bounds for axis 0 with size 90

The issue seem to be identical to #11 which I reported a while back. I recall not seeing the error right after you fixed it, but now the error is reappearing. I ran the experiment 3 times and each time only 100 scenarios are generated.

Questions:

1. Can you take a look at the bash command and confirm whether this is the right command to run ADFuzz in attempt to reproduce the experiment discussed in the paper?
2. Will you be able to look into the issue and fix it in the near future?

[AIasd]

Sorry for the late reply! 1.Yea, this seems right 2.Can you let me know what commit has this issue and what commit you tried does not? It looks like the problem is caused by some inconsistency between the index and the data

[YuqiHuai]

Hi @AIasd , thanks for getting back to me! I am using

0281022 (HEAD -> main, origin/main, origin/HEAD) add doc on integrating CARLA+OpenPilot from FusED; more testing is needed

Seems to be the latest commit.

---

sorry I missed the 2nd half of your questions earlier today. I will spend some time reverting to previous versions and check if there is a commit with no problem.