Adding agents mid simulation

[rusu24edward]

How can we add agents mid simulation? Some important factors to think about:

1. When we copy an agent, it copies the seed too, so the random actions will be exactly the same. We'll need to adjust the seed to allow for variety.
2. Where to place new agent in a non-overlapping grid?
3. Can the current agents dictionary infrastructure support adding agents?
   • Consider things like all done and the way the managers loop over the agents.
   • Encodings in sim will need to be adjusted
   • Most of the components rely on the encodings that are available at init. Adding/changing encodings will break most of these components.
   • If an observation is agent based (rather than grid based), then the observation spaces of potentially all the agents will grow mid-simulation. Is this even possible?
   • If the agent dict is modified during the simulation, what is the agent dict at the start of the next episode?
4. Can these new agents use policies that already exist? Does this screw up the training?
5. If new agents require their own policies, do the policies need to be pre-configured and added to the trainer before training starts? That is, can new policies spawn mid-episode?

[rusu24edward]

This should be considered together with #279