search

NVlabs / traffic-behavior-simulation

Other
176 stars 26 forks source link

Errors on evaluation of some scenes #2

Closed BestSonny closed 1 year ago

BestSonny commented 1 year ago

Any hint on fixing the error when running the following:

python scripts/evaluate.py --results_root_dir results/ --dataset_path nuscenes --env nusc --num_scenes_per_batch 1 --ckpt_yaml evaluation/BITS_example.yaml --eval_class HierAgentAware --render

I was trying to run evaluate more than two scenes (on the nuscenes dataset). And get the error when evaluating on the third scene. (I also get the error on 19th scene, i guess there might be more similar errors in other scenes).

Thank you very much.

Traceback (most recent call last): File "/blue/ranka/pan.he/tbsim/scripts/evaluate.py", line 427, in run_evaluation( File "/blue/ranka/pan.he/tbsim/scripts/evaluate.py", line 147, in run_evaluation stats, info, renderings, adjust_plans, trace = rollout_episodes( File "/blue/ranka/pan.he/tbsim/tbsim/utils/env_utils.py", line 286, in rollout_episodes action = policy.get_action(obs_torch, step_index=counter) File "/blue/ranka/pan.he/tbsim/tbsim/policies/wrappers.py", line 278, in get_action agents_action, agents_action_info = self.agents_policy.get_action( File "/blue/ranka/pan.he/tbsim/tbsim/policies/wrappers.py", line 234, in get_action action, action_info = self.policy.get_action(obs, kwargs) File "/blue/ranka/pan.he/tbsim/tbsim/policies/wrappers.py", line 175, in get_action return self.model.get_action(obs, self.action_kwargs, kwargs) File "/blue/ranka/pan.he/tbsim/tbsim/policies/wrappers.py", line 129, in get_action action_idx = ego_sample_planning( File "/blue/ranka/pan.he/tbsim/tbsim/utils/planning_utils.py", line 198, in ego_sample_planning current_lane = nearby_lanes[np.argmin(dis)] File "<__array_function__ internals>", line 200, in argmin File "/home/pan.he/.local/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 1338, in argmin return _wrapfunc(a, 'argmin', axis=axis, out=out, kwds) File "/home/pan.he/.local/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc return bound(*args, **kwds) ValueError: attempt to get argmin of an empty sequence

chenyx09 commented 1 year ago

Hi, it seems that when BITS is choosing from samples, the set of samples is empty, which is quite rare. One possible cause is you have some filter on (e.g., only sample from the drivable area) and the car is out of the drivable area.

BestSonny commented 1 year ago

@chenyx09 I found out that changing the code at https://github.com/NVlabs/traffic-behavior-simulation/blob/main/tbsim/utils/planning_utils.py#L188 from 30 to a larger value, e.g., 100 fixed the problem. Want to get a confirmation that it is a correct fix. It finds a minimal distance in the end so large values for filtering seems to be OK?

chenyx09 commented 1 year ago

Hi @BestSonny , that is definitely one way to fix it. Sometimes the vectorized map can behave weirdly and we probably need to add a few lines to handle the case where no lanes can be found.