search

facebookresearch / habitat-lab

A modular high-level library to train embodied AI agents across a variety of tasks and environments.
https://aihabitat.org/
MIT License
1.96k stars 491 forks source link

Create multiagent environment #6

Closed HighlyAuditory closed 5 years ago

HighlyAuditory commented 5 years ago

Expected Results

My goal is to create a scene where there are other agents moving around, an ego view percept these movements. Also, I would like to keep all the images from the ego view along with the trajectory and angles of the ego camera. Is the platform compatible with this kind of task? Any potential tutorial to look at?

Thanks!

abhiskk commented 5 years ago

@LlamaSi, this feature is currently not supported. We have multi-agent support in our future roadmap. Although there is definitely a pseudo multiagent experimentation setup that you can do:

  1. Have multiple environments setup similar to train_ppo.py.
  2. Have the multiple environment be setup inside the same house.
  3. In each environment you can have a single agent. You can run the multiple agents in parallel inside multiple environments but the same house (pseudo-multiagent).
  4. Note that currently there is no way for the multiple agents to see each other or physically interact with each other (collisions etc).
  5. Although you can setup interactions for the agents on your side (for example communication between the agents etc).
HighlyAuditory commented 5 years ago

@LlamaSi, this feature is currently not supported. We have multi-agent support in our future roadmap. Although there is definitely a pseudo multiagent experimentation setup that you can do:

  1. Have multiple environments setup similar to train_ppo.py.
  2. Have the multiple environment be setup inside the same house.
  3. In each environment you can have a single agent. You can run the multiple agents in parallel inside multiple environments but the same house (pseudo-multiagent).
  4. Note that currently there is no way for the multiple agents to see each other or physically interact with each other (collisions etc).

Thanks a lot for the answer! May I ask if a simpler case with one agent and multiple dynamic objects is possible? Like some mesh moving around inside the scene. Can the agent see those moving objects?

abhiskk commented 5 years ago

@LlamaSi at the moment it is not possible. A simple feature of moving mesh would go hand in hand with a better multiagent support. But this feature is not in place right now.

HighlyAuditory commented 5 years ago

Thanks!

xiaotaw commented 4 years ago

Looking forward to multi-agent support too. [smile]

@abhiskk's pseudo multiagent seems great enough to meet my needs tho.

srama2512 commented 4 years ago

@abhiskk - what would be the timeframe for enabling multi-agent support in Habitat? Can it be expected in the near future? Also, are there any plans to have a standard implementation of pseudo multiagent?

mathfac commented 4 years ago

Hi @srama2512! We will have team planning in near future and let you know.

smorad commented 3 years ago

Has there been any update on this? I see multi agent support exists many places in the codebase, but there are no guides on how to add additional agents. Adding AGENT_1, AGENT_2, ... to the habitat config doesn't work.

JiayunjieJYJ commented 3 years ago

Has there been any update on this? I see multi agent support exists many places in the codebase, but there are no guides on how to add additional agents. Adding AGENT_1, AGENT_2, ... to the habitat config doesn't work.

@smorad I have the same question, do you know how to create multi agent environment in habitat?