TypeError: 'NoneType' object is not subscriptable

[BingHan0458]

Hello, when I run ./gym_collision_avoidance/experiments/run_trajectory_dataset_creator.sh according to https://gym-collision-avoidance.readthedocs.io/en/latest/pages/use_cases.html#collect-a-dataset-of-trajectories, there is an error as follows:

Traceback (most recent call last):
  File "src/run_trajectory_dataset_creator.py", line 157, in <module>
    main()
  File "src/run_trajectory_dataset_creator.py", line 130, in main
    agents = test_case_fn(**test_case_args)
TypeError: get_testcase_random() got an unexpected keyword argument 'agents_policy'

So I try to replace agents = test_case_fn() with agents = test_case_fn(**test_case_args), but there is another error:

Traceback (most recent call last):
  File "src/run_trajectory_dataset_creator.py", line 158, in <module>
    main()
  File "src/run_trajectory_dataset_creator.py", line 140, in main
    times_to_goal, extra_times_to_goal, collision, all_at_goal, any_stuck, agents = run_episode(env, one_env)
  File "/home/hanbin/catkin_ws/src/CADRL/rl_collision_avoidance/gym-collision-avoidance/gym_collision_avoidance/experiments/src/env_utils.py", line 50, in run_episode
    obs, rew, done, info = env.step([None])
  File "/home/hanbin/catkin_ws/src/CADRL/rl_collision_avoidance/gym-collision-avoidance/baselines/baselines/common/vec_env/vec_env.py", line 108, in step
    return self.step_wait()
  File "/home/hanbin/catkin_ws/src/CADRL/rl_collision_avoidance/gym-collision-avoidance/baselines/baselines/common/vec_env/dummy_vec_env.py", line 51, in step_wait
    obs, self.buf_rews[e], self.buf_dones[e], self.buf_infos[e] = self.envs[e].step(action)
  File "/home/hanbin/catkin_ws/src/CADRL/rl_collision_avoidance/gym-collision-avoidance/venv/lib/python3.6/site-packages/gym/core.py", line 263, in step
    observation, reward, done, info = self.env.step(action)
  File "/home/hanbin/catkin_ws/src/CADRL/rl_collision_avoidance/gym-collision-avoidance/gym_collision_avoidance/envs/collision_avoidance_env.py", line 161, in step
    self._take_action(actions, dt)
  File "/home/hanbin/catkin_ws/src/CADRL/rl_collision_avoidance/gym-collision-avoidance/gym_collision_avoidance/envs/collision_avoidance_env.py", line 245, in _take_action
    all_actions[agent_index, :] = agent.policy.external_action_to_action(agent, actions[agent_index])
TypeError: 'NoneType' object is not subscriptable

How to solve it? Thank you very much!

[mfe7]

Good question, I haven't done a good job keeping the test_case_fn methods up to date as things have changed. I tried to automate the process of generating new test cases but it's really badly maintained, sorry about that.

In the dataset generation script, here the code sets the test case kwarg 'agents_policy' -- in the past this was supposed to be a Policy class, but now it should be a string (e.g., 'noncoop' or others defined in this dict. You also may also want to remove the for policy in policies: loop to simplify things -- I think it is supposed to take in a list of policies and generate trajectories for all of those policies.

regarding the nonetype error -- i suspect that is related to the test_case_fn, because that line should only be executed for agents who have an ExternalPolicy. because one of the agents has an externalpolicy (probably because that is how agents were initialized in test_case_fn, whose default policy argument is "learning" (an externalpolicy)), the line that gives the error is looking for an externally provided actions dict, but i doubt the dataset collection script or run_episode method provides that dict. You may want to start with all agents using a built-in policies, such as ga3c-cadrl, and then they will choose actions "internally" to the environment.

all of that to say: i think the error will go away if you replace L129 with test_case_args['policies'] = 'noncoop' and then clean up all the things about policies in run_trajectory_dataset_creator.py. Then you could try to add back in the relevant pieces to get a ga3c-cadrl policy to work.