No module named 'pyrep.robots.arms.dual_panda

[Yifei-Wei]

Hi , I have met this confusing problem . Traceback (most recent call last): File "dataset_generator.py", line 5, in from rlbench import ObservationConfig File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/rlbench/init.py", line 12, in from rlbench.environment import Environment File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/rlbench/environment.py", line 10, in from pyrep.robots.arms.dual_panda import PandaLeft ModuleNotFoundError: No module named 'pyrep.robots.arms.dual_panda'

that is no files dual_panda in pyrep , and i also check my package there is no dual_panda indeed . i also check the official repo there is also no dual_panda at all . See the screen-shot below. what is more there is no files that is about the dual things like dual_panda,dual gripper and so on . i wonder if pyrep have just chang the repo?

[Yifei-Wei]

@markusgrotz looking forward to your help. Appreciate.

[markusgrotz]

Hi Yifei-Wei,

Thank you very much for your interest. It seems like you are using the official PyRep. Can you please check if you are using my fork? The repository is here https://github.com/markusgrotz/Pyrep/ and instructions for installation are here https://github.com/markusgrotz/peract_bimanual/blob/main/INSTALLATION.md. I am working on a Docker image and I have script to install necessary dependencies (see the README)

Does this help you?

Kind regards, Markus

[Yifei-Wei]

thanks for your help. But there is another problem for running train.py (rlbench) yfwu@comp:~/bimanual/peract_bimanual$ python train.py train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") /home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing _self_. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information warnings.warn(msg, UserWarning) /home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/PERACT_BC': Usage of deprecated keyword in package header '# @package group'. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/changes_to_package_header for more information deprecation_warning( /home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default. See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information. ret = run_job( [2024-11-06 22:25:22,422][root][INFO] - method: name: PERACT_BC agent_type: leader_follower robot_name: bimanual image_crop_size: 64 bounds_offset:

• 0.15 voxel_sizes:
• 100 include_prev_layer: false num_latents: 2048 latent_dim: 512 transformer_depth: 6 transformer_iterations: 1 cross_heads: 1 cross_dim_head: 64 latent_heads: 8 latent_dim_head: 64 pos_encoding_with_lang: true conv_downsample: true lang_fusion_type: seq voxel_patch_size: 5 voxel_patch_stride: 5 final_dim: 64 low_dim_size: 4 input_dropout: 0.1 attn_dropout: 0.1 decoder_dropout: 0.0 lr: 0.0005 lr_scheduler: false num_warmup_steps: 3000 optimizer: lamb lambda_weight_l2: 1.0e-06 trans_loss_weight: 1.0 rot_loss_weight: 1.0 grip_loss_weight: 1.0 collision_loss_weight: 1.0 rotation_resolution: 5 activation: lrelu norm: None crop_augmentation: true transform_augmentation: apply_se3: true aug_xyz:
  • 0.125
  • 0.125
  • 0.125 aug_rpy:
  • 0.0
  • 0.0
  • 45.0 aug_rot_resolution: ${method.rotation_resolution} demo_augmentation: true demo_augmentation_every_n: 10 no_skip_connection: false no_perceiver: false no_language: false keypoint_method: heuristic ddp: master_addr: localhost master_port: '0' num_devices: 1 rlbench: task_name: multi tasks:
• open_drawer
• slide_block_to_color_target demos: 100 demo_path: /my/demo/path episode_length: 25 cameras:
• over_shoulder_left
• over_shoulder_right
• overhead
• wrist_right
• wrist_left
• front camera_resolution:
• 128
• 128 scene_bounds:
• -0.3
• -0.5
• 0.6
• 0.7
• 0.5
• 1.6 include_lang_goal_in_obs: true replay: batch_size: 8 timesteps: 1 prioritisation: false task_uniform: true use_disk: true path: /tmp/arm/replay max_parallel_processes: 32 framework: log_freq: 100 save_freq: 100 train_envs: 1 replay_ratio: ${replay.batch_size} transitions_before_train: 200 tensorboard_logging: true csv_logging: true training_iterations: 40000 gpu: 0 env_gpu: 0 logdir: /tmp/arm_test/ logging_level: 20 seeds: 1 start_seed: 0 load_existing_weights: true num_weights_to_keep: 60 num_workers: 0 record_every_n: 5 checkpoint_name_prefix: checkpoint

[2024-11-06 22:25:22,423][root][INFO] - CWD:/tmp/arm_test/multi/PERACT_BC [2024-11-06 22:25:22,423][root][INFO] - Starting seed 0. /home/yfwu/bimanual/peract_bimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") [11/06/24 22:25:23] INFO INFO:torch.distributed.distributed_c10d:Added key: store_based_barrier_key:1 to store for rank: 0 distributed_c10d.py:442 INFO INFO:torch.distributed.distributed_c10d:Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 1 nodes. distributed_c10d.py:476 INFO INFO:root:Using method PERACT_BC with type leader_follower agent_factory.py:24 Error executing job with overrides: [] Traceback (most recent call last): File "train.py", line 93, in main mp.spawn( File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 239, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 197, in start_processes while not context.join(): File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error: Traceback (most recent call last): File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/utils/versions.py", line 80, in require_version got_ver = pkg_resources.get_distribution(pkg).version File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 534, in get_distribution dist = get_provider(dist) File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 417, in get_provider return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 1070, in require needed = self.resolve(parse_requirements(requirements)) File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 897, in resolve dist = self._resolve_dist( File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 938, in _resolve_dist raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'sacremoses' distribution was not found and is required by the application

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap fn(i, *args) File "/home/yfwu/bimanual/peract_bimanual/run_seed_fn.py", line 45, in run_seed agent = agent_factory.create_agent(cfg) File "/home/yfwu/bimanual/peract_bimanual/agents/agent_factory.py", line 28, in create_agent agent_fn = agent_fn_by_name(method_name) File "/home/yfwu/bimanual/peract_bimanual/agents/agent_factory.py", line 92, in agent_fn_by_name from agents import peract_bc File "/home/yfwu/bimanual/peract_bimanual/agents/peract_bc/init.py", line 1, in import agents.peract_bc.launch_utils File "/home/yfwu/bimanual/peract_bimanual/agents/peract_bc/launch_utils.py", line 8, in from agents.peract_bc.qattention_peract_bc_agent import QAttentionPerActBCAgent File "/home/yfwu/bimanual/peract_bimanual/agents/peract_bc/qattention_peract_bc_agent.py", line 28, in import transformers File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/init.py", line 43, in from . import dependency_versions_check File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/dependency_versions_check.py", line 41, in require_version_core(deps[pkg]) File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/utils/versions.py", line 94, in require_version_core return require_version(requirement, hint) File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/utils/versions.py", line 82, in require_version raise pkg_resources.DistributionNotFound(requirement, ["this application", hint]) pkg_resources.DistributionNotFound: The 'sacremoses' distribution was not found and is required by this application, Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git master

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

[Yifei-Wei]

I think the main error is /home/yfwu/bimanual/peract_bimanual/train.py:18: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_name="config", config_path="conf") [11/06/24 22:25:23] INFO INFO:torch.distributed.distributed_c10d:Added key: store_based_barrier_key:1 to store for rank: 0 distributed_c10d.py:442 INFO INFO:torch.distributed.distributed_c10d:Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 1 nodes. distributed_c10d.py:476 INFO INFO:root:Using method PERACT_BC with type leader_follower agent_factory.py:24 Error executing job with overrides: [] Traceback (most recent call last): File "train.py", line 93, in main mp.spawn( File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 239, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 197, in start_processes while not context.join(): File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error: Traceback (most recent call last): File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/utils/versions.py", line 80, in require_version got_ver = pkg_resources.get_distribution(pkg).version File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 534, in get_distribution dist = get_provider(dist) File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 417, in get_provider return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 1070, in require needed = self.resolve(parse_requirements(requirements)) File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 897, in resolve dist = self._resolve_dist( File "/home/yfwu/.local/lib/python3.8/site-packages/pkg_resources/init.py", line 938, in _resolve_dist raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'sacremoses' distribution was not found and is required by the application

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yfwu/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap fn(i, *args) File "/home/yfwu/bimanual/peract_bimanual/run_seed_fn.py", line 45, in run_seed agent = agent_factory.create_agent(cfg) File "/home/yfwu/bimanual/peract_bimanual/agents/agent_factory.py", line 28, in create_agent agent_fn = agent_fn_by_name(method_name) File "/home/yfwu/bimanual/peract_bimanual/agents/agent_factory.py", line 92, in agent_fn_by_name from agents import peract_bc File "/home/yfwu/bimanual/peract_bimanual/agents/peract_bc/init.py", line 1, in import agents.peract_bc.launch_utils File "/home/yfwu/bimanual/peract_bimanual/agents/peract_bc/launch_utils.py", line 8, in from agents.peract_bc.qattention_peract_bc_agent import QAttentionPerActBCAgent File "/home/yfwu/bimanual/peract_bimanual/agents/peract_bc/qattention_peract_bc_agent.py", line 28, in import transformers File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/init.py", line 43, in from . import dependency_versions_check File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/dependency_versions_check.py", line 41, in require_version_core(deps[pkg]) File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/utils/versions.py", line 94, in require_version_core return require_version(requirement, hint) File "/home/yfwu/anaconda3/envs/rlbench/lib/python3.8/site-packages/transformers-4.3.2-py3.8.egg/transformers/utils/versions.py", line 82, in require_version raise pkg_resources.DistributionNotFound(requirement, ["this application", hint]) pkg_resources.DistributionNotFound: The 'sacremoses' distribution was not found and is required by this application, Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git master

[markusgrotz]

It seems like it is missing some dependencies. Can you trypip install transformers -U ?

[markusgrotz]

I have the transformers==4.21.2. I just checked the code for transformers 4.3 and sacremoses is a dependency. It seems like this was dropped in newer versions. Does this solve your issue?

[Yifei-Wei]

thanks . it do solves my problem. The environment problems always sucks me . I spent a whole night fighting with it . thanks for your help .

[Yifei-Wei]

@markusgrotz It is really kind of you . Thanks for your help . I think I need to go to bed because it is late enough here . Have a good day. Best wishes

[markusgrotz]

You are welcome! Setting up a coding environment takes time. I hope you can work on some cool research soon.

Regarding the training: Please don't forget to increase the batch size if you have more GPU memory available. Also PerACT performs better on multi-task. Training longer seems to help

[Yifei-Wei]

Hi , @markusgrotz . I am reading the paper and redo the task and I have a question about the prediction procedure. Does the whole project work like : lets say that 1 demo have 3 key points , and pass the nets , the nets will give the next 3 best positions. Then the robot move to the predict points step by step to finish the whole trail. My question is that whether the Peract only predicts several best points but not the whole trail like ALoha ACT did ?

[markusgrotz]

Yes ACT is a policy that predicts joint angles for each timestep, whereas PerACT predicts a Cartesian pose given a keyframe. Checkout Mohit's talk https://youtu.be/QcuXwmQgurE?t=2279

[Yifei-Wei]

@markusgrotz thanks for it .Will check it later.