search

stepjam / RLBench

A large-scale benchmark and learning environment.
https://sites.google.com/corp/view/rlbench
Other
1.11k stars 226 forks source link

Failed tasks while running dataset_generator.py #39

Closed nevakanezzar closed 4 years ago

nevakanezzar commented 4 years ago

I ran python3 dataset_generator.py --episodes_per_task 1 --processes 5

1) I can't tell if there was a speed-up or not from using multiple processes, but that's for me to investigate further at my end.

2) 4 out of those 5 processes errored out at some point on put_plate_in_colored_dish_rack:

i)

Process Process-2:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 81, in reset
    randomly_place=not self._static_positions)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/scene.py", line 97, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/put_plate_in_colored_dish_rack.py", line 56, in init_episode
    self.boundary.sample(self.dish_rack, min_distance=0.2)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/spawn_boundary.py", line 176, in sample
    'Could not place the object within the boundary due to '
rlbench.backend.exceptions.BoundaryError: Could not place the object within the boundary due to collision with other objects in the boundary.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 165, in run
    obs, descriptions = task_env.reset()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 86, in reset
    % self._task.get_name()) from e
rlbench.task_environment.TaskEnvironmentError: Could not place the task put_plate_in_colored_dish_rack in the scene. This should not happen, please raise an issues on this task.

ii)

Process Process-3:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 81, in reset
    randomly_place=not self._static_positions)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/scene.py", line 97, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/put_plate_in_colored_dish_rack.py", line 56, in init_episode
    self.boundary.sample(self.dish_rack, min_distance=0.2)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/spawn_boundary.py", line 176, in sample
    'Could not place the object within the boundary due to '
rlbench.backend.exceptions.BoundaryError: Could not place the object within the boundary due to collision with other objects in the boundary.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 165, in run
    obs, descriptions = task_env.reset()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 86, in reset
    % self._task.get_name()) from e
rlbench.task_environment.TaskEnvironmentError: Could not place the task put_plate_in_colored_dish_rack in the scene. This should not happen, please raise an issues on this task.

iii)

Process Process-4:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 81, in reset
    randomly_place=not self._static_positions)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/scene.py", line 97, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/put_plate_in_colored_dish_rack.py", line 56, in init_episode
    self.boundary.sample(self.dish_rack, min_distance=0.2)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/spawn_boundary.py", line 176, in sample
    'Could not place the object within the boundary due to '
rlbench.backend.exceptions.BoundaryError: Could not place the object within the boundary due to collision with other objects in the boundary.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 165, in run
    obs, descriptions = task_env.reset()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 86, in reset
    % self._task.get_name()) from e
rlbench.task_environment.TaskEnvironmentError: Could not place the task put_plate_in_colored_dish_rack in the scene. This should not happen, please raise an issues on this task.

iv)

Process Process-5:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 81, in reset
    randomly_place=not self._static_positions)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/scene.py", line 97, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/put_plate_in_colored_dish_rack.py", line 56, in init_episode
    self.boundary.sample(self.dish_rack, min_distance=0.2)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/spawn_boundary.py", line 176, in sample
    'Could not place the object within the boundary due to '
rlbench.backend.exceptions.BoundaryError: Could not place the object within the boundary due to collision with other objects in the boundary.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 165, in run
    obs, descriptions = task_env.reset()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 86, in reset
    % self._task.get_name()) from e
rlbench.task_environment.TaskEnvironmentError: Could not place the task put_plate_in_colored_dish_rack in the scene. This should not happen, please raise an issues on this task.
stepjam commented 4 years ago

Hi @nevakanezzar, Thanks for pointing out. Ill take a look at the put_plate_in_colored_dish_rack task.

Regarding your first point, number of processes should speed up the data collection if you are collecting more than one task. Each of the processes gets given a task and an associated variation to collect. These is a trade off: the more simulations you have running the slower each of them will run because of resource contention, but generally as long as you keep processes < num_cores, then it should be much faster :)

stepjam commented 4 years ago

Please let me know if V 1.0.5 fixes your issue. Best, Stephen

nevakanezzar commented 4 years ago

Hey Stephen,

Thanks for this. It has gotten better; it doesn't error out as frequently, yet, I still encountered the following, hence leaving this issue open.

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 81, in reset
    randomly_place=not self._static_positions)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/scene.py", line 97, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/put_plate_in_colored_dish_rack.py", line 56, in init_episode
    self.boundary.sample(self.dish_rack, min_distance=0.2)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/spawn_boundary.py", line 176, in sample
    'Could not place the object within the boundary due to '
rlbench.backend.exceptions.BoundaryError: Could not place the object within the boundary due to collision with other objects in the boundary.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 165, in run
    obs, descriptions = task_env.reset()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 86, in reset
    % self._task.get_name()) from e
rlbench.task_environment.TaskEnvironmentError: Could not place the task put_plate_in_colored_dish_rack in the scene. This should not happen, please raise an issues on this task.
nevakanezzar commented 4 years ago

A few more:

Process Process-6:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 81, in reset
    randomly_place=not self._static_positions)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/scene.py", line 99, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/put_knife_in_knife_block.py", line 31, in init_episode
    self.boundary.sample(self.chopping_board)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/spawn_boundary.py", line 176, in sample
    'Could not place the object within the boundary due to '
rlbench.backend.exceptions.BoundaryError: Could not place the object within the boundary due to collision with other objects in the boundary.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 182, in run
    obs, descriptions = task_env.reset()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 86, in reset
    % self._task.get_name()) from e
rlbench.task_environment.TaskEnvironmentError: Could not place the task put_knife_in_knife_block in the scene. This should not happen, please raise an issues on this task.
Process 0 collecting task: pour_from_cup_to_cup // variation: 4
Process Process-2:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 81, in reset
    randomly_place=not self._static_positions)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/scene.py", line 99, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/pour_from_cup_to_cup.py", line 51, in init_episode
    [b.sample(d, min_distance=0.12) for d in self.distractors]
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/pour_from_cup_to_cup.py", line 51, in <listcomp>
    [b.sample(d, min_distance=0.12) for d in self.distractors]
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/spawn_boundary.py", line 172, in sample
    raise BoundaryError('Could not place within boundary.'
rlbench.backend.exceptions.BoundaryError: Could not place within boundary.Perhaps the object is too big for it?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/user/alpha/taskemb/env/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 182, in run
    obs, descriptions = task_env.reset()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/task_environment.py", line 86, in reset
    % self._task.get_name()) from e
rlbench.task_environment.TaskEnvironmentError: Could not place the task pour_from_cup_to_cup in the scene. This should not happen, please raise an issues on this task.
stepjam commented 4 years ago

As we are now on V 1.0.8, I'll (optimistically) assume that these tasks now run fine.