Is it possible to use multiple robot?

[grlinnn]

Hi!

I'm still super new to Reinforcement Learning.

I want to know if it is possible to train multiple robots instead of just one like your code applied?

Thankyou!

[Shuijing725]

There are many ways to do this, I'll just share my thoughts.

• If you assume all robots share the same RL policy, then you just need to change the gym environment: we have to calculate the observation for n robots in the generate_ob function, and use the same network to output n actions, which are applied to all n robots in the step function.
• If you assume each robot has its own policy, then we need to apply the environment changes above, plus instantiate as many networks as you need, and train all of them instead of just one.

[grlinnn]

I'm honestly still not quite clear about how to add n more robots.

Can you tell me where exactly I can add more robot instances?

I was exploring the code for a while, and had a rough guess that I need add more Robot instances in crowd_sim/envs/crowd_sim.py such applied at self.robot2 = None and follow the other first robot's config.

Thank you.

[Shuijing725]

Sorry about the late reply. I agree with your idea of adding more robot instances in the gym environment. For the additional robots, you can follow the code of the first robot. The only difference (I think) is that each robot has different observations, dynamics, and actions.

[grlinnn]

Hi!

Recently I have tried to deal with this problem. I have adding one more robot instances in crowd_sim/envs/crowd_sim.py and crowd_sim/envs/crowd_sim_dict.py. I have done something like :

# set robot for this envs
        rob_RL = Robot(config, 'robot')
        rob2_RL = Robot(config,'robot2')
        self.set_robot(rob_RL)
        self.set_robot(rob2_RL)

and

ob['robot_node'] = self.robot.get_full_state_list_noV()
ob['robot_node2'] = self.robot2.get_full_state_list_noV()

Is this the correct way to do it? Since I am still having bugs although I have followed the first robot's code.

Sorry for bothering you, I am still curious about this thing.

Thankyou!

[Shuijing725]

Hi @grlinnn,

Based on what you provided, I think your implementation looks good. A minor problem is that set_robot function will also set the observation space of the environment, which is done twice and a little redundant. But I don't think it will cause a big issue, as long as you set both robot_RL and robot2_RL as self.robot and self.robot2.

[grlinnn]

Hi!

Sorry to bother you again.

I want to ask you if you have any idea on my recent problem that I can't overcome.

There are two lines of code appear on cmd:

ob, rew, done, info = self.env.step(action)

action = self.robot.policy.clip_action(action, self.robot.v_pref)

After I had some updates like I mentioned earlier in this post, the following problem occurred and I had no idea what's going on.

AttributeError: 'ORCA' object has no attribute 'clip_action'

Do you have any idea what probably cause this problem?

Thankyou!

[Shuijing725]

It's probably because you set the self.robot.policy to ORCA, but clip_action function is only implemented for "srnn" policy. Please see the files in crowd_nav/policy for reference.

[grlinnn]

After re-examining the code, I saw that robot.policy is set to several locations:

in train.py

actor_critic = Policy(
        envs.observation_space.spaces, # pass the Dict into policy to parse
        envs.action_space,
        base_kwargs=config,
        base=config.robot.policy)

in config.py

# robot settings
    robot = BaseConfig()
    robot.visible = False  # the robot is visible to humans
    # robot policy: srnn for now
    robot.policy = 'srnn'

in crowd_sim.py

# clip the action to obey robot's constraint
        action = self.robot.policy.clip_action(action, self.robot.v_pref)

in crowd_sim_dict.py action = self.robot.policy.clip_action(action, self.robot.v_pref)

There I set the robot.policy to 'srnn'. Therefore, I still meet a dead-end about what thing could probably cause that error.

Do you have any other idea about this?

Thankyou!

[Shuijing725]

I think the attribute error you mentioned last week is only related to the second policy you just mentioned in config.py. The third and fourth policy is defined in the same place in config.py.