Task Planning in Habitat without training RL policy

[RAraghavarora]

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: v0.3.0

Habitat-Sim: v0.3.0

Habitat is under active development, and we advise users to restrict themselves to stable releases. Are you using the latest release versions of Habitat-Lab and Habitat-Sim? Your question may already be addressed in the latest versions. We may also not be able to help with problems in earlier versions because they sometimes lack the more verbose logging needed for debugging.

Master branch contains 'bleeding edge' code and should be used at your own risk.

Docs and Tutorials

Did you read the docs? https://aihabitat.org/docs/habitat-lab/

Yes

Did you check out the tutorials? https://aihabitat.org/tutorial/2020/

Yes

Perhaps your question is answered there. If not, carry on!

❓ Questions and Help

Is it possible to use multi-agent setup in Habitat 3.0 social_nav without training any RL policy? I am unable to train the baseline policies on my GPU, and want to use Habitat combined with some PDDL based planner for human-robot collaboration. I am also on headless HPC, so am unable to interactively execute actions.

For configs like hssd_spot_human.yaml, the action space has 25 dimensions, and the value of each dimension is not very clear. Is it possible to make the agents move around and perform some actions without a trained policy?

Thank you in advance!

[RAraghavarora]

I see that the action keys are:

agent_0_base_velocity:Dict(agent_0_base_vel:Box(-20.0, 20.0, (2,), float32)), 
agent_1_base_velocity:Dict(agent_1_base_vel:Box(-20.0, 20.0, (2,), float32)), 
agent_1_oracle_nav_action:Dict(agent_1_oracle_nav_action:Box(-3.4028235e+38, 3.4028235e+38, (1,), float32)), 
agent_1_oracle_nav_randcoord_action:Dict(agent_1_oracle_nav_randcoord_action:Box(-3.4028235e+38, 3.4028235e+38, (1,), float32)), 
agent_1_pddl_apply_action:Dict(agent_1_pddl_action:Box(-3.4028235e+38, 3.4028235e+38, (18,), float32)), 
agent_1_rearrange_stop:EmptySpace())

Using basic logic, I am able to move agent_0 with the 1st 2 elements of the action. However, agent_1 does not seem to be moving despite changing any of the other action values. (I check it using observation 'agent_1_head_rgb')

What could be the reason for this?

Could you also clarify what are the actions 'pddl_apply_action' and 'oracle_nav_action' used for?

Thanks in advance

[mannytheshizzle]

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: v0.3.0

Habitat-Sim: v0.3.0

Habitat is under active development, and we advise users to restrict themselves to stable releases. Are you using the latest release versions of Habitat-Lab and Habitat-Sim? Your question may already be addressed in the latest versions. We may also not be able to help with problems in earlier versions because they sometimes lack the more verbose logging needed for debugging.

Master branch contains 'bleeding edge' code and should be used at your own risk.

Docs and Tutorials

Did you read the docs? https://aihabitat.org/docs/habitat-lab/

Yes

Did you check out the tutorials? https://aihabitat.org/tutorial/2020/

Yes

Perhaps your question is answered there. If not, carry on!

❓ Questions and Help

Is it possible to use multi-agent setup in Habitat 3.0 social_nav without training any RL policy? I am unable to train the baseline policies on my GPU, and want to use Habitat combined with some PDDL based planner for human-robot collaboration. I am also on headless HPC, so am unable to interactively execute actions.

For configs like hssd_spot_human.yaml, the action space has 25 dimensions, and the value of each dimension is not very clear. Is it possible to make the agents move around and perform some actions without a trained policy?

Thank you in advance!

can anyone assist with this? I am also trying to figure out how to perform some actions without a trained policy

[mannytheshizzle]

@RAraghavarora did you manage to figure this out? I'd appreciate any pointers you might have in this regard

[RAraghavarora]

Hi @mannytheshizzle No, I couldn't figure it out. It seems having a trained policy is a requirement for the experiments. They released a checkpoint for socialnav

I ultimately gave up though, moved to other simulation environments..

[xavierpuigf]

Sorry for the late reply here. While we don't have a task planner ready, it is possible to call habitat with actions without having a policy. Therefore, you could connect these actions with a planner, fixed script or LLM to run the agents, instead of a pretrained policy.

In this tutorial we show some examples to how to call actions: https://github.com/facebookresearch/habitat-lab/blob/main/examples/tutorials/articulated_agents_tutorial.ipynb

Note that the example uses oracle_nav_coord_actions, which uses a path planner to decide the steps to reach a given coordinate. With that and pick+place actions you should be able to do rearrangement tasks without a pretrained policy.

Let me know if this helps!