Communication as component

[rusu24edward]

Add the communication component.

In Epic #12

[rusu24edward]

Communication: Communication is a subset of observations, a way to augment your observations with that of another agent. Typically we think of communication dynamics being point-to-point or distance-based. We can try to implement both of these within the grid observation framework.

Distance based: the agent will have a communication range that should be greater than its view. The agent sees the environment according to its view. Beyond that, the agent sees a "fog" until the end of its communication distance. Observations from other agents can fill in the fog with real data. We will need to modify the other agents observations in two ways, however. First, the obs from the sending agent must be translated to the correct location for the receiving agent's observation. Second, the obs from the sending agent may need to be clipped to fit within the receiving agent's total box. For example, suppose m and a have a view of 2, and m has a total distance of 5. Suppose a is near m and an observation is exchanged. The following ascii art shows us that a's observation, indicated with 1s, will be offset and partially clipped because the last column is outside of a's total distance.

+ - - - - - - - - - - - +
| f f f f f f f f f f f |
| f f f f f f f f f f f |
| f f f f f f f f f f f |
| f f f 0 0 0 0 0 f f f |
| f f f 0 0 0 0 0 f f f |
| f f f 0 0 m 0 0 f f f + - +
| f f f 0 0 0 0 2 1 1 1 | 1 |
| f f f 0 0 0 0 2 1 1 1 | 1 |
| f f f f f f f 1 1 a 1 | 1 |
| f f f f f f f 1 1 1 1 | 1 |
| f f f f f f f 1 1 1 1 | 1 |
+ - - - - - - - - - - - + - +

[rusu24edward]

Following up on the above grid obs + communication comment, an easier path would be to use a fixed spatial observation, where all the agents observe the entire map. Most of the map is "fog" except for the patch near the agent. Then, when an observation is shared, a patch of the map is "unfogged" for that step.

Another route to go down is to used distance-based observations, and only support observing other agents, no resources. Then, communication just appends/fills the observation list.

[rusu24edward]

Observations from other agents can just be appended to the padded observation space. We need to pre-calculate the max space size depending on the communication channels.

398 also has us talking about combining observations from multiple agents. This is pretty straightforward using the new AbsoluteGridObserver.