Multi-Agent Environments

[kayuksel]

Hello,

Are you planning to create any multi-agent environment such as crowd simulation?

Is there also possibility to have a non-uniform terrain, walls etc in each environment?

so that each agent can be initialized in a random location for varying its experience.

(without that, I don't see a major advantage of parallel simulation capability of engine)

Sincerely, Kamer

[cdfreeman-google]

Hello!

Yes, we do have plans for multi-agent environments, so stay tuned! And yes, we also have some support for heightmaps that was recently added by a wonderful contributor. Check out https://github.com/google/brax/blob/main/brax/physics/colliders.py#L111 . You could also create this sort of variability by randomizing the scene across a batch. If you have a certain sort of randomization in mind that you don't think is supported, we can try to see what the delta might be to add it.

[kayuksel]

@cdfreeman-google Looking forward to play with them once they're ready! :) I don't have any specific randomization method in mind, I guess that randomizing agent location in each member of a batch would be sufficient as long as terrain is not uniform.

[kayuksel]

@cdfreeman-google Just thinking, procedural level (e.g. height-map) generation for games could be a great use-case for Brax.

[erikfrey]

Hello! We have some examples of multi-agent training in the braxlines project. Check out the environments such as sumo or squidgame in the experimental braxlines composer colab.

[erikfrey]

Closing this as I think it's addressed, but please do re-open if you have any remaining questions.

[ishitamed19]

Hi @cdfreeman-google , I wanted to know if procedural generation is possible in any of the environments in Brax? Like is there any API through which, in every episode, we can customise how the environment and the objects in it look like, or are placed in different configs?

[cdfreeman-google]

The short answer is "sort of" :)

Due to Jax compilation requirements, the training loop has to be recompiled if the config of the system changes. That doesn't mean, however, that you can't do whatever you want to the system in the mean time. It should be possible to, for example, define a template config with parts that you programmatically enable/disable (though this is slightly annoying to do right now with how configs are ingested by systems).

If you have something concrete in mind, I can try to describe how this might work!

[ishitamed19]

For example, if I I wanted to try introducing some sort of curriculum using the Reacher environment. Then, based on the progress so far, I want to be able to modify how many objects are there in the current episode (multiple targets instead of single target), and their textures. So in each episode, I want to be able to modify the number of objects in the scene and each object's individual properties (like color, texture, shape, etc.).

[cdfreeman-google]

So, the way I would do something like that depends a little bit on the variability of objects in question, but would look something like this:

If number of objects ~ 10s:

1. Create a config with every object that the reacher might interact with over a full curriculum.
2. Design the reacher's observation space to be number-of-object agnostic (have it explicitly look at only properties for the nearest object or something like that)
3. Put curriculum logic in the reset method that puts the current "difficulty" of object in the range of the reacher, and moves the others far out of range where they won't effect the simulation

If number of object is like ~1000s because you would be sampling things like masses / lengths for certain primitives (like radii or box edge lengths or something), then you should be able to accomplish that with the previous method with some careful bookkeeping, assuming those 1000s of objects can be generated from 10s of primitives

If you want 1000s of actually completely distinct objects that can't be templated easily, then that's not easily supported right now.