vwxyzjn
commented
2 years ago Support multiple maps during training,
As we discussed, the naïve way of supporting this feature is to reinitialize the vectorized environment, along with the storage variables such as obs, actions, every once in a while.
However, the most desirable way I think is to swap the map after a sub-environment in the vectorized environment is done.
For instance, if 5th index in done in the following code is True,
Then, we know the 5th environment is done. We can then do something slightly hacky like calling
responses = self.vec_client.gameStep(self.actions, [0] * self.num_envs)
reward, done = np.array(responses.reward), np.array(responses.done)
obs = [self._encode_obs(np.array(ro)) for ro in responses.observation]
infos = [{"raw_rewards": item} for item in reward]
# reset the 5th environment with a new map
if done[:, 0][4] == Ture:
self.vec_client.clients[4].mapPath = desired_new_map_path
response = self.vec_client.clients[4].reset()
obs[4] = self._encode_obs(np.array(response.observation))
return np.array(obs), reward @ self.reward_weight, done[:, 0], infos
There are several steps for this goal: