Issues with evaluation

[klaus99815]

Thank you for your amazing work!

I have followed INSTALL.md to setup the environment . When I runROUTES=assets/routes_lav_valid.xml ./leaderboard/scripts/run_evaluation.sh , the following error occurs.

========= Preparing RouteScenario_0 (repetition 0) =========

Setting up the agent

Could not set up the required agent:

invalid load key, 'v'.

Traceback (most recent call last): File "/home/abc/carla1/LAV-main/leaderboard/leaderboard/leaderboard_evaluator.py", line 262, in _load_and_run_scenario self.agent_instance = getattr(self.module_agent, agent_class_name)(args.agent_config) File "/home/abc/carla1/LAV-main/leaderboard/leaderboard/autoagents/autonomous_agent.py", line 45, in init self.setup(path_to_conf_file) File "/home/abc/carla1/LAV-main/team_code/lav_agent.py", line 89, in setup bev_planner.load_state_dict(torch.load(self.bev_model_dir)) File "/home/abc/anaconda3/envs/LAV-env/lib/python3.7/site-packages/torch/serialization.py", line 713, in load return _legacy_load(opened_file, map_location, pickle_module, pickle_load_args) File "/home/abc/anaconda3/envs/LAV-env/lib/python3.7/site-packages/torch/serialization.py", line 920, in _legacy_load magic_number = pickle_module.load(f, pickle_load_args) _pickle.UnpicklingError: invalid load key, 'v'.

Registering the route statistics

ps: import torch torch.version '1.11.0'

any help will be appreciated!

[klaus99815]

Thank you for your amazing work!

I have followed INSTALL.md to setup the environment . When I runROUTES=assets/routes_lav_valid.xml ./leaderboard/scripts/run_evaluation.sh , the following error occurs.

========= Preparing RouteScenario_0 (repetition 0) =========

Setting up the agent

Could not set up the required agent:

invalid load key, 'v'.

Traceback (most recent call last): File "/home/abc/carla1/LAV-main/leaderboard/leaderboard/leaderboard_evaluator.py", line 262, in _load_and_run_scenario self.agent_instance = getattr(self.module_agent, agent_class_name)(args.agent_config) File "/home/abc/carla1/LAV-main/leaderboard/leaderboard/autoagents/autonomous_agent.py", line 45, in init self.setup(path_to_conf_file) File "/home/abc/carla1/LAV-main/team_code/lav_agent.py", line 89, in setup bev_planner.load_state_dict(torch.load(self.bev_model_dir)) File "/home/abc/anaconda3/envs/LAV-env/lib/python3.7/site-packages/torch/serialization.py", line 713, in load return _legacy_load(opened_file, map_location, pickle_module, pickle_load_args) File "/home/abc/anaconda3/envs/LAV-env/lib/python3.7/site-packages/torch/serialization.py", line 920, in _legacy_load magic_number = pickle_module.load(f, pickle_load_args) _pickle.UnpicklingError: invalid load key, 'v'.

Registering the route statistics

ps: import torch torch.version '1.11.0'

any help will be appreciated!

I think git-lfs is not installed in my environment. after install it, I just try again.But it doesn't work. So,I downloaded bev_160.th bra_10.th`` lidar_15.th seg_1.th uniplanner_15.th manually,and replaced the contents of the original weight file .

The above error did not appear.But there are new errors.

Traceback (most recent call last): File "/home/abc/carla1/LAV-main/leaderboard/leaderboard/scenarios/scenario_manager.py", line 152, in _tick_scenario ego_action = self._agent() File "/home/abc/carla1/LAV-main/leaderboard/leaderboard/autoagents/agent_wrapper.py", line 75, in call return self._agent() File "/home/abc/carla1/LAV-main/leaderboard/leaderboard/autoagents/autonomous_agent.py", line 115, in call control = self.run_step(input_data, timestamp) File "/home/abc/anaconda3/envs/LAV-env/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/home/yko/carla1/LAV-main/team_code/lav_agent.py", line 165, in run_step self.stop_counter += 1 AttributeError: 'LAVAgent' object has no attribute 'stop_counter'

I checked the code and found that there was no such attribute.

Is there anything else I haven't noticed?any help will be appreciated!

[Kin-Zhang]

I think git-lfs is not installed in my environment. after install it, I just try again.But it doesn't work. So,I downloaded bev_160.th bra_10.th`` lidar_15.th seg_1.th uniplanner_15.th manually,and replaced the contents of the original weight file .

Yes I also noticed this one, you should download the files through GitHub by downloading button.

AttributeError: 'LAVAgent' object has no attribute 'stop_counter'

Add initial for self.stop_counter is fine. please see the pull request: https://github.com/dotchen/LAV/pull/4

After all things are prepared, it should be fine to run the evaluation. For example here:

[klaus99815]

I think git-lfs is not installed in my environment. after install it, I just try again.But it doesn't work. So,I downloaded bev_160.th bra_10.th`` lidar_15.th seg_1.th uniplanner_15.th manually,and replaced the contents of the original weight file .

Yes I also noticed this one, you should download the files through GitHub by downloading button.

AttributeError: 'LAVAgent' object has no attribute 'stop_counter'

Add initial for self.stop_counter is fine. please see the pull request: #4

After all things are prepared, it should be fine to run the evaluation. For example here:

Thank you for your help! I can run now. I have another problem. When I run, the image of the simulator is no longer smooth. I try to use print() in the_tick_scenario(self, timestamp) function in scenario_manager.py, and the output speed slows down. Is this normal? It hasn't appeared before when running WorldOnRails.

[Kin-Zhang]

Thank you for your help! I can run now. I have another problem. When I run, the image of the simulator is no longer smooth. I try to use print() in the_tick_scenario(self, timestamp) function in scenario_manager.py, and the output speed slows down. Is this normal? It hasn't appeared before when running WorldOnRails.

I think you should check the CPU and GPU utility when you run this to see whether your computer can run it at a high rate. I believe this model may be larger than world on rail.

check CPU: htop

check GPU: watch nvidia-smi

[dotchen]

Please make sure you have git-lfs installed before cloning the repo.

[klaus99815]

Thank you for your help! I can run now. I have another problem. When I run, the image of the simulator is no longer smooth. I try to use print() in the_tick_scenario(self, timestamp) function in scenario_manager.py, and the output speed slows down. Is this normal? It hasn't appeared before when running WorldOnRails.

I think you should check the CPU and GPU utility when you run this to see whether your computer can run it at a high rate. I believe this model may be larger than world on rail.

check CPU: htop

check GPU: watch nvidia-smi

Hi,I checked the CPU and GPU.It looks so strange...

I think it shouldn't take up so much CPU resources. Is there something wrong? Or just this model is larger than world on rail,my machine can't provide better support... PS:

CPU:Intel® Xeon(R) CPU E5-2630 v4 @ 2.20GHz × 40 GPU:NVIDIA GeForce RTX 2080 Ti

[Kin-Zhang]

CPU:Intel® Xeon(R) CPU E5-2630 v4 @ 2.20GHz × 40 GPU:NVIDIA GeForce RTX 2080 Ti

I believe this CPU is a server CPU and normally servers' CPU is not so strong for processing things before data goes to GPU, maybe you can try it at your computer. The server normally is for training and using GPU mainly.

Here is my personal computer running for evaluation:

And the computer setting is:

Processor: Intel® Core™ i7-9750H CPU @ 2.60GHz × 12 GPU: GeForce GTX 1660 Ti/PCIe/SSE2