How to run OVMM RL agent with categorical/discrete actions

[trangml-mit]

🐛 Bug

When training an RL agent with categorical actions, the habitat_baselines code throws an AttributeError: 'Box' object has no attribute 'n' What is the proper way to run RL training to generate an RL agent with specific allowed_actions? Additionally, once setting it to categorical, how do we put the allowed_actions in the trainer so that it knows that during RL?

Steps to Reproduce

python -u -m habitat_baselines.run \ --exp-config habitat-baselines/habitat_baselines/config/ovmm/rl_skill.yaml \ --run-type train benchmark/ovmm=<skill_name> \ habitat_baselines.checkpoint_folder=data/new_checkpoints/ovmm/<skill_name>

Change the yaml config for running RL agent from "gaussian" to "categorical"

  rl:
    policy:
        name: "PointNavResNetPolicy"
        action_distribution_type: "categorical"

Steps to reproduce the behavior:

Traceback (most recent call last):
....
  File "habitat-baselines/habitat_baselines/run.py", line 102, in <module>
    main()
  File "habitat-baselines/habitat_baselines/run.py", line 54, in main
    run_exp(**vars(args))
  File "habitat-baselines/habitat_baselines/run.py", line 98, in run_exp
    execute_exp(config, run_type)
  File "habitat-baselines/habitat_baselines/run.py", line 81, in execute_exp
    trainer.train()
  File "/home/ma32004/mambaforge/envs/home-robot/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/ma32004/home-robot/src/third_party/habitat-lab/habitat-baselines/habitat_baselines/rl/ppo/ppo_trainer.py", line 765, in train
    self._init_train(resume_state)
  File "/home/ma32004/home-robot/src/third_party/habitat-lab/habitat-baselines/habitat_baselines/rl/ppo/ppo_trainer.py", line 318, in _init_train
    self._setup_actor_critic_agent(ppo_cfg)
  File "/home/ma32004/home-robot/src/third_party/habitat-lab/habitat-baselines/habitat_baselines/rl/ppo/ppo_trainer.py", line 149, in _setup_actor_critic_agent
    self.actor_critic = policy.from_config(
  File "/home/ma32004/home-robot/src/third_party/habitat-lab/habitat-baselines/habitat_baselines/rl/ddppo/policy/resnet_policy.py", line 124, in from_config
    return cls(
  File "/home/ma32004/home-robot/src/third_party/habitat-lab/habitat-baselines/habitat_baselines/rl/ddppo/policy/resnet_policy.py", line 78, in __init__
    PointNavResNetNet(
  File "/home/ma32004/home-robot/src/third_party/habitat-lab/habitat-baselines/habitat_baselines/rl/ddppo/policy/resnet_policy.py", line 300, in __init__
    action_space.n + 1, self._n_prev_action
AttributeError: 'Box' object has no attribute 'n'

Expected behavior

An RL agent with discrete actions to be trained, similar to the pretrained agents provided from the benchmark.

[yvsriram]

Hi,

In addition to switching to categorical, you will also need to change the allowed actions here to the discrete set:

- stop
- move_forward
- turn_left
- turn_right

[trangml-mit]

After switching to that as well, I get the following error

hydra.errors.ConfigCompositionException: In 'habitat/task/ovmm/nav_to_rec': ConfigKeyError raised while composing config:
Key 'navmesh_offset' not in 'ActionConfig'
    full_key: habitat.task.actions.base_velocity.navmesh_offset
    reference_type=ActionConfig
    object_type=ActionConfig

Which I assume is due to these actions set here.

actions:
  base_velocity:
    type: "BaseWaypointTeleportAction"
    navmesh_offset: [[0.0, 0.0]]

What's the correct replacement for this section of the yaml file?

[yvsriram]

Yes, you need to delete those 4 lines. Needn't replace with anything.