Replicable-MARL / MARLlib

One repository is all that is necessary for Multi-agent Reinforcement Learning (MARL)
https://marllib.readthedocs.io
MIT License
879 stars 142 forks source link

JointQ not work in custom env #223

Open Yilgrimage opened 7 months ago

Yilgrimage commented 7 months ago

i build a new env on marllib, and i test it by MAPPO, it worked. but when I try to switch algo to qmix, it met a problem in preprocessors.py. it occurs in "check_shape", self._obs_space is a tuple of GymDict when I use JointQ algos, which should be a GymDict I guess. ,it makes " if not self._obs_space.contains(observation):"(ray/rllib/models/preprocessors.py", line 93)"return false and raise ValueError. I don't know why smac doesn't meet this problem, I believe I used the right form to build my env. This is my obs_space,and I ensure "step" and "reset" will return right data.

self.observation_space = GymDict({
            "obs": Box(
                low=0,
                high=10,
                shape=(self.obs_size,),
                dtype=np.dtype("int64")),
            "state": Box(
                low=-1,
                high=self.n_agents+self.n_preys*10,
                shape=((self.map_size + 2 * self.sight_range)**2,),
                dtype=np.dtype("int64")),
            "action_mask": Box(
                low=0,
                high=1,
                shape=(self.n_actions,),
                dtype=np.dtype("int64"))

        })
Yilgrimage commented 6 months ago

I get it. when you build your obs dict in step or reset, you should ensure the key of obs dict is a member of self.agents. If not, it will happen a problem in GroupWrapper, which doesn't affect some algorithms, but algorithms like QMIX can be problematic.