Using Camera observation for training

[ozgraslan]

Hi,

When I tried to use camera observation with: python3 -m rl.main --env FurnitureBaxterBlockEnv --prefix demo --wandb False --reward_scale 3 --debug True --notes True --save_rollout True --unity False --is_train True --visual_ob True command it gives this error:

RuntimeError: invalid argument 0: Tensors must have same number of dimensions: got 3 and 1 at /pytorch/aten/src/TH/generic/THTensor.cpp:603

How can I use camera observation while training? Thanks in advance

[edwhu]

Which github branch are you using? Try switching to dev branch and see if this works.

[ozgraslan]

I was using master branch. When I tried the same command with dev branch, I got a different error:

File "/home/deneme/furniture-dev/env/furniture.py", line 1379, in _reset logger.debug(f"{body} {pos_init[i]} {quat_init[i]}") IndexError: list index out of range

[edwhu]

I looked into the problem. There was a bug with FurnitureBaxterBlockEnv loading an incorrect furniture. Also, our actor critic is currently an MLP architecture. We will add a CNN architecture for the next release. I pushed the changes to dev branch.

Do you want to use camera observation as input to the policy? If you do, I recommend downsizing the image width and height since by default it is 500x500.

Here is the command I ran to ensure RL is working.

python3 -m rl.main --env FurnitureBaxterBlockEnv --prefix demo --wandb False --reward_scale 3 --debug True --notes True --save_rollout True --unity False --is_train True --visual_ob False --furniture_name block --screen_width 24 --screen_height 24

[ozgraslan]

When I tried the above command with the latest development environment, I got this error: New ob_norm with shape {'object_ob': [14], 'robot_ob': [28]} Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/deneme/furniture-dev/rl/main.py", line 123, in <module> run(args) File "/home/deneme/furniture-dev/rl/main.py", line 69, in run trainer = Trainer(config) File "/home/deneme/furniture-dev/rl/trainer.py", line 83, in __init__ config, ob_space, ac_space, actor, critic File "/home/deneme/furniture-dev/rl/sac_agent.py", line 40, in __init__ self._critic1 = critic(config, ob_space, ac_space) File "/home/deneme/furniture-dev/rl/policies/mlp_actor_critic.py", line 115, in __init__ input_dim = sum([np.prod(x) for x in ob_space.values()]) AttributeError: 'Dict' object has no attribute 'values'

[edwhu]

Fixed. 7b146ed

[ozgraslan]

Thank you so much.