search

absdnd / aux_distill

2 stars 0 forks source link

AttributeError: 'habitat_sim._ext.habitat_sim_bindings.ManagedBulle' object has no attribute 'creation_attributes' #2

Open upqsdy46853 opened 1 month ago

upqsdy46853 commented 1 month ago

I want to test the results of the pretrained weight, but when I execute bash scripts/eval/rearrange_pretrained.sh, the following error message appears.

$bash scripts/eval/rearrange_pretrained.sh Num tasks: 6 Max skills: 11 Force-Terminate -1.0 Using Dataset: data/datasets/replica_cad/rearrange/v1/val/rearrange_hard_200.json.gz Observation Keys: ['obj_start_sensor', 'obj_goal_sensor', 'head_depth', 'joint', 'ee_pos', 'is_holding', 'relative_resting_position', 'env_task_id_sensor', 'is_art_sensor', 'hier_skill_sensor'] wandb: (1) Create a W&B account wandb: (2) Use an existing W&B account wandb: (3) Don't visualize my results wandb: Enter your choice: 3 wandb: You chose "Don't visualize my results" wandb: Tracking run with wandb version 0.16.6 wandb: W&B syncing is set to offline in this directory.
wandb: Run wandb online or set WANDB_MODE=online to enable cloud syncing. Observation Keys: ['obj_start_sensor', 'obj_goal_sensor', 'head_depth', 'joint', 'ee_pos', 'is_holding', 'relative_resting_position', 'env_task_id_sensor', 'is_art_sensor', 'hier_skill_sensor'] Evaluating Checkpoint: data/ckpts/pretrained/rearrange_seed_120.pth 2024-07-24 19:37:15,814 Loaded checkpoint trained for 480709104 steps Allowed_Skills ['nav_pick_nav_place'] Num Environments: 1 Num Skills: 1 2024-07-24 19:37:15,832 Initializing dataset RearrangeDataset-v0 2024-07-24 19:37:17,611 Initializing dataset RearrangeDataset-v0 2024-07-24 19:37:17,689 initializing sim RearrangeSim-v0 MeshTools::compile(): ignoring Trade::MeshAttribute::TextureCoordinates 1 as its biding slot is already occupied by Trade::MeshAttribute::TextureCoordinates 0 MeshTools::compile(): ignoring Trade::MeshAttribute::TextureCoordinates 1 as its biding slot is already occupied by Trade::MeshAttribute::TextureCoordinates 0 MeshTools::compile(): ignoring Trade::MeshAttribute::TextureCoordinates 1 as its biding slot is already occupied by Trade::MeshAttribute::TextureCoordinates 0 MeshTools::compile(): ignoring Trade::MeshAttribute::TextureCoordinates 1 as its biding slot is already occupied by Trade::MeshAttribute::TextureCoordinates 0 2024-07-24 19:37:19,161 Initializing task CustomRearrangePddlTask-v0 2024-07-24 19:37:19,502 Distillation loss coefficient will not be increased: Setting maximum value immediately 2024-07-24 19:37:19,503 Number of params to train: 12996301 2024-07-24 19:37:19,504 Agent number of parameters: 12996301 Process ForkServerProcess-1: Traceback (most recent call last): File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/multiprocessing/process.py", line 108, in run self._target(self._args, self._kwargs) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/core/vector_env.py", line 268, in _worker_env observations = env.reset() File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/gym/gym_env_episode_count_wrapper.py", line 61, in reset return self.env.reset(kwargs) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/gym/gym_env_obs_dict_wrapper.py", line 36, in reset return self.env.reset(kwargs) File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/gym/core.py", line 283, in reset return self.env.reset(kwargs) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/gym/gym_wrapper.py", line 319, in reset obs = self.env.reset(args, return_info=return_info, kwargs) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/core/environments.py", line 62, in reset return super().reset(*args, return_info=return_info, *kwargs) File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/contextlib.py", line 79, in inner return func(args, kwds) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/core/env.py", line 425, in reset observations = self._env.reset() File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/core/env.py", line 261, in reset self.reconfigure(self._config) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/core/env.py", line 343, in reconfigure self._sim.reconfigure(self._config.simulator, self.current_episode) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/tasks/rearrange/utils.py", line 718, in wrapper ret = f(self, *args, *kwargs) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/tasks/rearrange/rearrange_sim.py", line 333, in reconfigure self._add_objs(ep_info, should_add_objects, new_scene) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/tasks/rearrange/utils.py", line 718, in wrapper ret = f(self, args, **kwargs) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/tasks/rearrange/rearrange_sim.py", line 648, in _add_objs self._receptacles = self._create_recep_info( File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/tasks/rearrange/rearrange_sim.py", line 668, in _create_recep_info all_receps = find_receptacles( File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/datasets/rearrange/samplers/receptacle.py", line 758, in find_receptacles creation_attr = obj.creation_attributes AttributeError: 'habitat_sim._ext.habitat_sim_bindings.ManagedBulle' object has no attribute 'creation_attributes' wandb:
wandb: You can sync this run to the cloud by running: wandb: wandb sync /home/pochih/Desktop/aux_distill/wandb/offline-run-20240724_193714-v350ltrm wandb: Find logs at: ./wandb/offline-run-20240724_193714-v350ltrm/logs Error executing job with overrides: ['habitat_baselines.writer_type=wb', 'habitat_baselines.wb.project_name=skill-chain', 'habitat_baselines.wb.entity=skillchainz', 'habitat_baselines.trainer_name=aux_trainer', 'habitat_baselines.wb.run_name=rearrange', 'habitat_baselines.num_environments=24', 'habitat_baselines.total_num_steps=5.0e8', 'habitat_baselines.checkpoint_folder=data/ckpts/rearrange', 'habitat_baselines.rl.ddppo.backbone=resnet50', '+habitat_baselines.rl.policy.main_agent.use_one_hot_critic=False', '+habitat_baselines.rl.ppo.use_norm_returns=True', 'habitat_baselines.rl.ppo.lr=0.0003', '+habitat_baselines.rl.policy.main_agent.use_mask_sensors=True', '+habitat_baselines.rl.policy.main_agent.use_task_specific_head=True', '+habitat_baselines.rl.policy.main_agent.use_pop_art_norm=True', '+habitat_baselines.rl.policy.main_agent.beta_decay=0.0003', 'habitat.task.lab_sensors.env_task_sensor.max_skills=11', 'env_tasks.skills.nav_pick_nav_place.update_rl_params.gamma=0.999', 'env_tasks.allowed_skills=[nav_pick_nav_place]', 'habitat_baselines.evaluate=True', 'habitat_baselines.load_resume_state_config=False', 'habitat_baselines.num_environments=1', 'habitat.dataset.data_path=data/datasets/replica_cad/rearrange/v1/val/rearrange_hard_200.json.gz', 'habitat.task.measurements.force_terminate.max_instant_force=-1.0', 'habitat.task.measurements.force_terminate.max_accum_force=-1.0', 'env_tasks.skills.nav_pick_nav_place.update_measures.task.measurements.force_terminate.max_instant_force=-1.0', 'habitat_baselines.video_dir=data/videos/rearrange/', 'habitat_baselines.eval_ckpt_path_dir=data/ckpts/rearrange/latest.pth', 'habitat.gym.obs_keys=[obj_start_sensor,obj_goal_sensor,head_depth,joint,ee_pos,is_holding,relative_resting_position,env_task_id_sensor,is_art_sensor,hier_skill_sensor]', 'habitat_baselines.wb.run_name=rearrange_eval', 'habitat_baselines.eval_ckpt_path_dir=data/ckpts/pretrained/rearrange_seed_120.pth'] Traceback (most recent call last): File "/home/pochih/Desktop/aux_distill/skill_chain/run.py", line 52, in main() File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/main.py", line 94, in decorated_main _run_hydra( File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra _run_app( File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/_internal/utils.py", line 457, in _run_app run_and_report( File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/_internal/utils.py", line 223, in run_and_report raise ex File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/_internal/utils.py", line 220, in run_and_report return func() File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/_internal/utils.py", line 458, in lambda: hydra.run( File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/internal/hydra.py", line 132, in run = ret.return_value File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/core/utils.py", line 260, in return_value raise self._return_value File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/site-packages/hydra/core/utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) File "/home/pochih/Desktop/aux_distill/skill_chain/run.py", line 45, in main trainer.eval() File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-baselines/habitat_baselines/common/base_trainer.py", line 129, in eval self._eval_checkpoint( File "/home/pochih/Desktop/aux_distill/skill_chain/trainer/aux_trainer.py", line 362, in _eval_checkpoint super()._eval_checkpoint(checkpoint_path, writer, checkpoint_index) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-baselines/habitat_baselines/rl/ppo/ppo_trainer.py", line 889, in _eval_checkpoint evaluator.evaluate_agent( File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-baselines/habitat_baselines/rl/ppo/habitat_evaluator.py", line 48, in evaluate_agent observations = envs.reset() File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/core/vector_env.py", line 389, in reset results.append(read_fn()) File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/core/vector_env.py", line 110, in call res = self.read_fn() File "/home/pochih/Desktop/aux_distill/habitat-lab/habitat-lab/habitat/utils/pickle5_multiprocessing.py", line 68, in recv buf = self.recv_bytes() File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/multiprocessing/connection.py", line 216, in recv_bytes buf = self._recv_bytes(maxlength) File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/multiprocessing/connection.py", line 414, in _recv_bytes buf = self._recv(4) File "/home/pochih/micromamba/envs/hab_sc/lib/python3.9/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError

absdnd commented 1 month ago

Hello, before running the command could you write export HABITAT_ENV_DEBUG=1. It'll reveal the underlying issue beneath the EOFError which is displayed currently.

hatimwen commented 2 weeks ago

Hi @upqsdy46853 , I also met this issue and just solved it by reinstalling habitat-sim with conda install habitat-sim withbullet -c conda-forge -c aihabitat-nightly.

Ref: https://github.com/facebookresearch/habitat-lab/issues/1751#issuecomment-1899969496