Path in GA-UN-NN-GRAD

[asa7007]

python ga_fuzzing.py -p 2021 --n_gen 15 --pop_size 50 -r 'town07_front_0' -c 'go_straight_town07' --algorithm_name nsga2-un --has_run_num 700 --objective_weights -1 1 1 0 0 0 0 0 0 0 --rank_mode adv_nn --warm_up_path --warm_up_len 500 --check_unique_coeff 0 0.1 0.5 --has_display 0 --record_every_n_step 5 --only_run_unique_cases 1 --record_every_n_step 5 --debug 0

Which path needs to be given here?

and what is its format

[AIasd]

it can be left empty or a relative path of a result folder (got by running any algorithms).

[asa7007]

When I am running first fuzzing algorithm i.e # GA-UN

python ga_fuzzing.py -p 2015 --n_gen 2 --pop_size 2 -r 'town07_front_0' -c 'go_straight_town07' --algorithm_name nsga2-un --has_run_num 4 --objective_weights -1 1 1 0 0 0 0 0 0 0 --check_unique_coeff 0 0.1 0.5 --record_every_n_step 5 --debug 0

I am getting the following output. Please suggest what needs to be done

Traceback (most recent call last): File "carla_lbc/carla_specific_utils/carla_specific.py", line 582, in run_carla_simulation_helper leaderboard_evaluator = LeaderboardEvaluator(arguments, statistics_manager, launch_server, episode_max_time) File "carla_lbc/leaderboard/leaderboard/fuzzing.py", line 140, in init gpu = port_to_gpu(int(args.port)) File "/home/adas/Documents/self-driving-cars/customized_utils.py", line 200, in port_to_gpu gpu = port % n ZeroDivisionError: integer division or modulo by zero Exception ignored in: <function LeaderboardEvaluator.del at 0x7fbaaaf2edc0> Traceback (most recent call last): File "carla_lbc/leaderboard/leaderboard/fuzzing.py", line 205, in del self._cleanup(True) File "carla_lbc/leaderboard/leaderboard/fuzzing.py", line 219, in _cleanup self.client.stop_recorder() AttributeError: 'LeaderboardEvaluator' object has no attribute 'client' Process Process-8: Traceback (most recent call last): File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/adas/Documents/self-driving-cars/ga_fuzzing.py", line 135, in fun objectives, run_info = run_simulation(x, fuzzing_content, fuzzing_arguments, sim_specific_arguments, dt_arguments, launch_server, counter, port) File "carla_lbc/carla_specific_utils/carla_specific.py", line 477, in run_carla_simulation return run_carla_simulation_helper(customized_data, File "carla_lbc/carla_specific_utils/carla_specific.py", line 588, in run_carla_simulation_helper del leaderboard_evaluator UnboundLocalError: local variable 'leaderboard_evaluator' referenced before assignment

returned_data is missing

get job result for 3

[AIasd]

seems to be caused by gpu issue. Make sure your machine has at least one GPU (with at least 6GB memory). Also when you installed packages like tensorflow / torch, make sure the gpu version is installed.

[asa7007]

After installing the drivers related to GPU, and running the first fuzzing algorithm i,e # GA-UN , I am getting the following output. So, this is expected output right?

We have found 0 bugs in total. LowLevelFatalError [File:Unknown] [Line: 102] Exception thrown: close: Bad file descriptor Signal 11 caught. Malloc Size=65538 LargeMemoryPoolOffset=65554 CommonUnixCrashHandler: Signal=11 Malloc Size=65535 LargeMemoryPoolOffset=131119 Malloc Size=120416 LargeMemoryPoolOffset=251552 Engine crash handling finished; re-raising signal 11 for the default handler. Good bye. -------------------- kill server at port 2015 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2015 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2015 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2015 Killed /home/adas/.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

[AIasd]

Seems to be as expected. what does the content in the result folder look like?

[asa7007]

And while running the second algorithm GA-UN-NN-GRAD, I am getting the error as no GPU is available. But my system is already having a Tesla T4 GPU. Kindly suggest what needs to be done here. I am attaching the content of the error which i am getting

after mating len 0 300 self.n_offsprings 300

len(self.all_pop_run_X) 131 len(initial_X) 431 process_X finished (131, 92) (131,) Traceback (most recent call last): File "ga_fuzzing.py", line 1525, in run_ga_general(fuzzing_arguments, sim_specific_arguments, fuzzing_content, run_simulation) File "ga_fuzzing.py", line 1348, in run_ga_general run_ga(fuzzing_arguments, sim_specific_arguments, fuzzing_content, run_simulation) File "ga_fuzzing.py", line 1271, in run_ga algorithm.solve() File "pymoo/pymoo/model/algorithm.py", line 211, in solve self._solve(self.problem) File "pymoo/pymoo/model/algorithm.py", line 291, in _solve self.next() File "pymoo/pymoo/model/algorithm.py", line 263, in next self._next() File "ga_fuzzing.py", line 920, in _next self.set_off() File "ga_fuzzing.py", line 822, in set_off clf = train_net(X_train, y_train, [], [], batch_train=200, device_name=self.device_name) File "fuzzing_utils/pgd_attack.py", line 720, in train_net model.to(device) File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/torch/nn/modules/module.py", line 989, in to return self._apply(convert) File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/torch/nn/modules/module.py", line 641, in _apply module._apply(fn) File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/torch/nn/modules/module.py", line 664, in _apply param_applied = fn(param) File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/torch/nn/modules/module.py", line 987, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/torch/cuda/init.py", line 229, in _lazy_init torch._C._cuda_init() RuntimeError: No CUDA GPUs are available -------------------- kill server at port 2021 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2021 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2021 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2021 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2021

[asa7007]

In the second algorithm, how many iterations are there in total?

Ref: [get job result for 292]

[asa7007]

num_of_high_conf_examples 3.0 / 50

[asa7007]

get job result for 699

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ len(cur_X_remaining) 246 cur X filtering: 246 -> 49 {'counter': 700, 'has_run': 700, 'time_elapsed': 22631.606442451477, 'num_of_bugs': 246, 'num_of_unique_bugs': 49, 'num_of_interested_unique_bugs': 49, 'bugcounts and unique bug counts': [246, 0, 0, 0, 49, 0, 0, 0], 'mean_objectives_this_generation': [0.0, 2.637464105652283, 0.0, 0.622974082827568, 0.2899625599384308, 0.2985366544491185, 0.0, 0.0, 0.0, 0.0], 'current_F': array([[-0.03960776], [-0.06410127], [-0.05210117], [ 0.16328753]])} ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

n_infills / mating_max_iterations 1 / 40 len(off) 0 remaining_inds before 20 Mating finds 4 offsprings after doing 1 / 40 mating iterations Mating cannot generate new springs, terminate earlier. self.tmp_off 4 We have found 246 bugs in total. -------------------- kill server at port 2018 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2018 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2018 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2018 /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or kill -l [exitstatus] -------------------- kill server at port 2018 Killed

[asa7007]

Why am I getting this error?

python carla_lbc/rerun_and_data_analysis/rerun_scenario.py --parent_folder path_to_the_parent_folder_to_rerun

2022-12-26 17:22:17.971912: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-12-26 17:22:18.568371: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory 2022-12-26 17:22:18.568454: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory 2022-12-26 17:22:18.568476: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. Traceback (most recent call last): File "carla_lbc/rerun_and_data_analysis/rerun_scenario.py", line 45, in from carla_lbc.carla_specific_utils.carla_specific import run_carla_simulation, get_event_location_and_object_type, check_bug, get_unique_bugs, get_if_bug_list File "./carla_lbc/carla_specific_utils/carla_specific.py", line 31, in from leaderboard.fuzzing import LeaderboardEvaluator File "carla_lbc/leaderboard/leaderboard/fuzzing.py", line 47, in from srunner.scenarios.control_loss import * File "carla_lbc/scenario_runner/srunner/scenarios/control_loss.py", line 20, in from srunner.scenariomanager.scenarioatomics.atomic_behaviors import ChangeNoiseParameters, ActorTransformSetter File "carla_lbc/scenario_runner/srunner/scenariomanager/scenarioatomics/atomic_behaviors.py", line 31, in from agents.navigation.basic_agent import BasicAgent, LocalPlanner File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/init.py", line 22, in from . import scripts File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/init.py", line 21, in from . import train File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/train.py", line 33, in from agents.scripts import configs File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/configs.py", line 26, in from agents.scripts import networks File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/networks.py", line 30, in tfd = tf.contrib.distributions AttributeError: module 'tensorflow' has no attribute 'contrib'

[asa7007]

/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/networks.py --> Replaced tfd = tf.contrib.distributions with tf.compat.v1.estimator [Ref: https://www.datasciencelearner.com/attributeerror-module-tensorflow-has-no-attribute-contrib-solved/] . After that, getting the below error for python carla_lbc/rerun_and_data_analysis/rerun_scenario.py --parent_folder path_to_the_parent_folder_to_rerun

2022-12-27 11:15:58.700757: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-12-27 11:15:59.447794: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory 2022-12-27 11:15:59.447875: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory 2022-12-27 11:15:59.447892: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. Traceback (most recent call last): File "carla_lbc/rerun_and_data_analysis/rerun_scenario.py", line 47, in from carla_lbc.carla_specific_utils.carla_specific import run_carla_simulation, get_event_location_and_object_type, check_bug, get_unique_bugs, get_if_bug_list File "./carla_lbc/carla_specific_utils/carla_specific.py", line 31, in from leaderboard.fuzzing import LeaderboardEvaluator File "carla_lbc/leaderboard/leaderboard/fuzzing.py", line 47, in from srunner.scenarios.control_loss import * File "carla_lbc/scenario_runner/srunner/scenarios/control_loss.py", line 20, in from srunner.scenariomanager.scenarioatomics.atomic_behaviors import ChangeNoiseParameters, ActorTransformSetter File "carla_lbc/scenario_runner/srunner/scenariomanager/scenarioatomics/atomic_behaviors.py", line 31, in from agents.navigation.basic_agent import BasicAgent, LocalPlanner File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/init.py", line 22, in from . import scripts File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/init.py", line 21, in from . import train File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/train.py", line 33, in from agents.scripts import configs File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/configs.py", line 26, in from agents.scripts import networks File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/agents/scripts/networks.py", line 37, in class CustomKLDiagNormal(tfd.MultivariateNormalDiag): File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/tensorflow/python/util/lazy_loader.py", line 59, in getattr return getattr(module, item) File "/home/adas/.pyenv/versions/3.8.5/lib/python3.8/site-packages/tensorflow/python/util/module_wrapper.py", line 232, in _getattr attr = getattr(self._tfmw_wrapped_module, name) AttributeError: module 'tensorflow_estimator.python.estimator.api._v1.estimator' has no attribute 'MultivariateNormalDiag'

[AIasd]

both seem to be related to tensorflow version. what version of tensorflow you used?

[AIasd]

In the second algorithm, how many iterations are there in total?

Ref: [get job result for 292]

--n_gen specifies the number of generations --has_run_num specifies the number of scenarios at most to run

[asa7007]

2.11.0

PS: Tensorflow is installed along with the module named agents.

[asa7007]

Can you please help in resolving this error

[AIasd]

The tensorflow version might be the cause. I used 1.13.2.

[asa7007]

okay, but in the requirements.txt tensorflow is not getting installed. Can you please suggest the installation procedure for Tensorflow 1.13.2 since now by pip install tensorflow directly tensorflow is getting installed above 2.x version

[AIasd]

Try

pip install tensorflow-gpu==1.13.2

[asa7007]

pip install tensorflow-gpu==1.13.2

This error :-- ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.13.2 (from versions: 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.6.5, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.7.1, 2.7.2, 2.7.3, 2.7.4, 2.8.0rc0, 2.8.0rc1, 2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.9.0rc0, 2.9.0rc1, 2.9.0rc2, 2.9.0, 2.9.1, 2.9.2, 2.9.3, 2.10.0rc0, 2.10.0rc1, 2.10.0rc2, 2.10.0rc3, 2.10.0, 2.10.1, 2.11.0rc0, 2.11.0rc1, 2.11.0rc2, 2.11.0, 2.12.0) ERROR: No matching distribution found for tensorflow-gpu==1.13.2