Modificated RLBench doesn't match the format of peract-1 rlbench dataset

[shihao1895]

Hi, authors.

Thank you for your excellent work!

I am encountering an issue with the modified RLBench (https://github.com/markusgrotz/RLBench) as it does not match the format of the PerAct-1 RLBench dataset (https://drive.google.com/drive/folders/0B2LlLwoO3nfZfkFqMEhXWkxBdjJNNndGYl9uUDQwS1pfNkNHSzFDNGwzd1NnTmlpZXR1bVE?resourcekey=0-jRw5RaXEYRLe2W6aNrNFEQ).

Here is the error information I am receiving:

Traceback (most recent call last): File "/data/miniconda3/envs/rlbench/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/data/miniconda3/envs/rlbench/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/data/code/peract_bimanual/agents/replay_utils.py", line 559, in fill_replay demo = rlbench_utils.get_stored_demos( File "/data/code/peract_bimanual/libs/RLBench/rlbench/utils.py", line 138, in get_stored_demos obs[i].perception_data[f"{camera_name}_rgb"] = image AttributeError: 'Observation' object has no attribute 'perception_data' It appears that the data input format of PerAct-1 and PerAct-2 modified RLBench are not the same. Specifically, the utils.py files differ between the two versions:

PerAct-2 modified RLBench: https://github.com/markusgrotz/RLBench/blob/main/rlbench/utils.py PerAct-1 modified RLBench: https://github.com/MohitShridhar/RLBench/blob/peract/rlbench/utils.py Does the PerAct-2 code not support training with PerAct-1 and RVT-1/2 datasets?

Additionally, is the PerAct-2 RLBench fully functional now?

Thank you for your assistance!

[markusgrotz]

Thank you for the kind words and for bringing this to our attention. The modified version of RLBench supports two robots, whereas the original version was designed for a single robot.

To help with compatibility, I have code that converts the data back to the old RLBench format by specifying the robot. Let me know if you are interested.

I'm currently working on documentation to address these kinds of questions and will also be sharing more code snippets soon. I think a good documentation makes a huge difference. Please stay tuned for updates.

[shihao1895]

Hi, Markusgrotz,

Thank you for your respond and help !

Could you provide me with the code you mentioned that is compatible with both rlbench1 and rlbench2?

And I discovered another issue: rlbench2's replay cannot recover from disk. By the way, can you upload a generated replay to https://dataset.cs.washington.edu/ ?

Thank you!

[markusgrotz]

Hi shihao1895,

I am working on an extended code release, but I would like to get the documentation done first.

The replay feature is not part of PerAct, but was added to RVT.

If you need this feel free to create a pull request. In my case it was sufficient to generate the replay for each job, so I don't have it.

I hope this helps. Let me know if you have any further questions.

[shihao1895]

Thank you.