KeyError: 'lang_emb' and TypeError: catching classes that do not inherit from BaseException is not allowed

[PuzhenYuan]

Hi there, sorry to raise some of my confusion with you. I am using the branch robocasa and want to give script run_trained_agents.py a try. Could you please help me handle this error below?

My command line is

python run_trained_agents.py --agent /home/ypz/expdata/robocasa/im/bc_xfmr/08-11-None/seed_123_ds_human-50/20240811204559/models/model_epoch_20.pth --n_rollouts 5 --horizon 400 --seed 0 --render --camera_names robot0_frontview

With config and NN architecture successfully printed on the terminal, the Error message is

Traceback (most recent call last):
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/scripts/run_trained_agents_multitask.py", line 117, in rollout
    act = policy(ob=obs)
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/algo/algo.py", line 684, in __call__
    ac = self.policy.get_action(obs_dict=ob, goal_dict=goal)
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/algo/bc.py", line 785, in get_action
    output = self.nets["policy"](obs_dict, actions=None, goal_dict=goal_dict)
  File "/home/ypz/anaconda3/envs/robocasa/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/models/policy_nets.py", line 1326, in forward
    out = self.forward_train(obs_dict=obs_dict, actions=actions, goal_dict=goal_dict)
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/models/policy_nets.py", line 1278, in forward_train
    outputs = MIMO_Transformer.forward(self, **forward_kwargs)
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/models/obs_nets.py", line 1093, in forward
    transformer_inputs = TensorUtils.time_distributed(
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/utils/tensor_utils.py", line 951, in time_distributed
    outputs = op(**inputs, **kwargs)
  File "/home/ypz/anaconda3/envs/robocasa/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/models/obs_nets.py", line 475, in forward
    self.nets[obs_group].forward(inputs[obs_group])
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/models/obs_nets.py", line 256, in forward
    x = self.obs_nets[k](x, lang_emb=obs_dict[LANG_EMB_KEY])
KeyError: 'lang_emb'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/scripts/run_trained_agents_multitask.py", line 374, in <module>
    run_trained_agent(args)
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/scripts/run_trained_agents_multitask.py", line 234, in run_trained_agent
    stats, traj = rollout(
  File "/home/ypz/msclab/robocasa_space/robomimic/robomimic/scripts/run_trained_agents_multitask.py", line 158, in rollout
    except env.rollout_exceptions as e:
TypeError: catching classes that do not inherit from BaseException is not allowed

[icecap360]

Same issue. Seems like the robocasa updates were applied to the training scripts, but not the evaluation script.

[hahn5]

Hi, I've also been facing a problem with run_trained_agent.py that may be related. When running:

python run_trained_agent.py --agent /home/fabian/expdata/robocasa/im/diffusion_policy/08-06-10-bis-30/seed_123_ds_30-demos-normal/20240806014556/models/model_epoch_60.pth

I get this error:

Traceback (most recent call last):
  File "/home/fabian/research/robomimic/robomimic/scripts/run_trained_agent.py", line 374, in <module>
    run_trained_agent(args)
  File "/home/fabian/research/robomimic/robomimic/scripts/run_trained_agent.py", line 207, in run_trained_agent
    env, _ = FileUtils.env_from_checkpoint(
  File "/home/fabian/research/robomimic/robomimic/utils/file_utils.py", line 499, in env_from_checkpoint
    env = EnvUtils.create_env_from_metadata(
  File "/home/fabian/research/robomimic/robomimic/utils/env_utils.py", line 228, in create_env_from_metadata
    env = create_env(
  File "/home/fabian/research/robomimic/robomimic/utils/env_utils.py", line 172, in create_env
    env = env_class(
  File "/home/fabian/research/robomimic/robomimic/envs/env_robosuite.py", line 94, in __init__
    self.env = robosuite.make(self._env_name, **kwargs)
  File "/home/fabian/research/robosuite/robosuite/environments/base.py", line 41, in make
    return REGISTERED_ENVS[env_name](*args, **kwargs)
  File "/home/fabian/research/robocasa/robocasa/environments/kitchen/single_stage/kitchen_microwave.py", line 63, in __init__
    super().__init__(*args, **kwargs)
  File "/home/fabian/research/robocasa/robocasa/environments/kitchen/kitchen.py", line 137, in __init__
    super().__init__(
  File "/home/fabian/research/robosuite/robosuite/environments/manipulation/manipulation_env.py", line 182, in __init__
    super().__init__(
  File "/home/fabian/research/robosuite/robosuite/environments/robot_env.py", line 227, in __init__
    super().__init__(
  File "/home/fabian/research/robosuite/robosuite/environments/base.py", line 158, in __init__
    self._reset_internal()
  File "/home/fabian/research/robocasa/robocasa/environments/kitchen/single_stage/kitchen_microwave.py", line 78, in _reset_internal
    super()._reset_internal()
  File "/home/fabian/research/robocasa/robocasa/environments/kitchen/kitchen.py", line 606, in _reset_internal
    self._pre_action(action, policy_step)
  File "/home/fabian/research/robosuite/robosuite/environments/robot_env.py", line 598, in _pre_action
    robot.control(robot_action, policy_step=policy_step)
  File "/home/fabian/research/robosuite/robosuite/robots/wheeled_robot.py", line 121, in control
    self.composite_controller.set_goal(action)
  File "/home/fabian/research/robosuite/robosuite/controllers/composite/composite_controller.py", line 139, in set_goal
    controller.set_goal(action, update_wrt_origin=update_wrt_origin)
  File "/home/fabian/research/robosuite/robosuite/controllers/arm/osc.py", line 274, in set_goal
    self.goal_origin_to_eef_ori = self.compute_goal_orientation(  # set_goal_orientation
  File "/home/fabian/research/robosuite/robosuite/controllers/arm/osc.py", line 338, in compute_goal_orientation
    raise NotImplementedError
NotImplementedError

[snasiriany]

Hi all, we need to tie some loose ends, but at the moment the best way to run evaluation is to uncomment this section of the code: https://github.com/ARISE-Initiative/robomimic/blob/9a4b448cd4eb1d1a4a8f97e2c1c51717782b397f/robomimic/scripts/config_gen/bc_xfmr_gen.py#L54-L89

And fill in your provided checkpoint and paths.

@hahn5 regarding the error you are facing, there should be a fix for this if you pull the latest code in the robosuite repo (using the existing branch robocasa_v0.1)

[snasiriany]

There is now a dedicated script for evaluation: https://github.com/ARISE-Initiative/robomimic/blob/robocasa/robomimic/scripts/config_gen/eval_ckpt.py

This is the recommended way to run evaluations on checkpoints.