MarkovVectorEnv does not set is_vector_env = True

[singh-jayant]

I was trying to set up a pettingzoo multi-agent environment for use as a vectorized environment through the wrapper supersuit.pettingzoo_env_to_vec_env_v1 and I realized that MarkovVectorEnv does not set ìs_vector_env to True. Is this a design choice since this isn't a "real" multi-env class ? Or a bug? Or am I missing something?

Here are my package versions:

gymnasium==0.28.1
PettingZoo==1.22.3
SuperSuit==3.8.0

[elliottower]

I'm not 100% familiar with Gymnasium's vector environments, but I believe the distinction is that the vector environments in SuperSuit are intended to be used concatenated together (concat_vec_env), where they will return a list of observations, list of infos, etc. In the Gymnasium vector environments I believe they do the vectorization on the numpy level, meaning they will add a new dimension. So for example, I believe reward turns into rewards and gets indexed by the environment number. My guess is that it's more efficient to do it with the array itself, and simpler because it's ultimately just a single environment, rather than a messy amalgamation of multiple environments

We intend to copy over the functionality of Gymnasium's vector environments to PettingZoo natively at some point, as it would be cleaner to have that versus these wrappers which aren't tested in the same CI as PettingZoo (so we could very easily mess something up in the future). It's much harder to maintain two separate libraries, so these wrappers will all eventually be moved and most likely changed quite a bit to be more efficient, etc.

[elliottower]

Running all of the tests locally they still pass with the is_vector_env attribute set to True, but the behavior with various Gymnasium wrappers will likely change. Will test a bit more to see. Just For what it's worth, as an example with pistonball if you concat 1 vector env the first dimension is 20 (20 agents), with two the first dimension is 40 (20 agents * 2) and so on. The env.reset_infos which is returned on reset instead of info (see https://github.com/Farama-Foundation/SuperSuit/pull/226) is a list of length 20 with an empty dict for each agent. The base environment has infos indexed by the agent name and is a dict.

So the point is basically that the wrapper flattens the environments and removes the names of agents.

[elliottower]

Just did some testing with that attribute added and it then when you try to use Gymnasium wrappers it throws errors for missing the attributes single_observation_space etc. You can use them as is currently though, for example the normalization one:

from pettingzoo.butterfly import pistonball_v6
import supersuit as ss
import numpy as np
env = pistonball_v6.parallel_env()
env = ss.pettingzoo_env_to_vec_env_v1(env)
env = ss.concat_vec_envs_v1(env, 10, base_class="gymnasium")
obs, info = env.reset()
print("Pre-normalization maximum: ", np.amax(obs)) # 221

from gymnasium.wrappers import normalize
env = normalize.NormalizeObservation(env)
obs, info = env.reset()
print("Post-normalization maximum: ", np.amax(obs)) # 0.009999897608091765

[elliottower]

Adding a test to ensure that these work, so thanks for the heads up about this.