data/humanoids/humanoid_data/walking_motion_processed.pkl not exist

amiryanj commented 3 months ago

Error

I finally managed to run the script examples/interactive_play.py and now I can control the fetch robot. But when I try to control the humanoid I face the following error:

File "habitat-lab/habitat-lab/habitat/articulated_agent_controllers/humanoid_rearrange_controller.py", line 53, in __init__
    raise RuntimeError(
RuntimeError: Path does data/humanoids/humanoid_data/walking_motion_processed.pkl not exist. 
Reach out to the paper authors to obtain this data.
python-BaseException

Please advise if I can find this data somewhere (e.g. huggingface?) or I need to contact the authors for the data?

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: v0.3.1 Habitat-Sim: v0.3.1

Other comment

I also think both of the following arguments need to passed for interactive_play with the humanoid: python examples/interactive_play.py --control-humanoid --use-humanoid-controller

xavierpuigf commented 3 months ago

Hi, we need to update the instructions for interactive play, we will be doing this soon! The motion files are per avatar, so you can find each one underr the folder of the avatar you want to control: Here is one: data/humanoids/humanoid_data/female_2/female_2_motion_data_smplx.pkl

amiryanj commented 3 months ago

Thanks @xavierpuigf

I changed the following line: https://github.com/facebookresearch/habitat-lab/blob/5e9efaa67d54b3edf88ac304b4db7d8673278500/examples/interactive_play.py#L82

to DEFAULT_POSE_PATH = data/humanoids/female_0/female_0_motion_data_smplx.pkl"

now that is solved but, there is a new error:

Traceback (most recent call last):
  File "habitat-lab/examples/interactive_play.py", line 805, in <module>
    play_env(env, args, config)
  File "/habitat-lab/examples/interactive_play.py", line 499, in play_env
    humanoid_controller.reset(env._sim.articulated_agent.base_pos)
  File "habitat-lab/habitat-lab/habitat/articulated_agent_controllers/humanoid_base_controller.py", line 82, in reset
    self.prev_orientation = base_transformation.transform_vector(
AttributeError: only four-component swizzles are supported at most
corrupted size vs. prev_size

I also notice the class HumanoidBaseController is written by you. In fact, reset() function expects a mn.Matrix4 but I see a Vector3 is being passed to it, i.e. env._sim.articulated_agent.base_pos!

Any idea?

amiryanj commented 3 months ago

I also faced a few other errors (like variables not initiated), and just managed to add temporal workarounds and move forward. (I will share later). But still wondering where is the human. I want to make sure I'm running the example correctly. Is there any reference or tutorial for this @ASzot (as I see the example is written by you)?

xavierpuigf commented 3 months ago

@amiryanj. Thanks for the note. We should deprecate or update interactive play. We will be using a new interactive mode soon. Let me try to help with this though.

The humanoid controller should expect the base transformation, not the base position. Could you try doing:

humanoid_controller.reset(env.sim.articulated_agent.base_transformation)

You can see in the humanoid tutorial that this is how we reset it. https://github.com/facebookresearch/habitat-lab/blob/main/examples/tutorials/humanoids_tutorial.ipynb

facebookresearch / habitat-lab