how to generate gloabl rgba-view state in multi-agent context

[fmxFranky]

In marl context, common-used CTDE(centralized training decentralized excution) methods(e.g. MAPPO, QMIX) need not only individual observations but also global state of the multi-agent system, how to get the rgba view of the state in the current multi-agent interface? Thanks a lot!

[JacopoPan]

You can simply use PyBullet's:

[w, h, rgb, dep, seg] = p.getCameraImage(width=self.VID_WIDTH,
                                                     height=self.VID_HEIGHT,
                                                     shadow=1,
                                                     viewMatrix=self.CAM_VIEW,
                                                     projectionMatrix=self.CAM_PRO,
                                                     renderer=p.ER_TINY_RENDERER,
                                                     flags=p.ER_SEGMENTATION_MASK_OBJECT_AND_LINKINDEX,
                                                     physicsClientId=self.CLIENT
                                                     )

in any subclass of BaseAviary to get a single frame from the main rendered camera view and use this to create a new observations space that includes it, if that's what you want to do for your approach.

MultiAgenAviary has an observations space that is made of dictionaries comprising of all individual agents' observations at each time step.