Multiple observations error

[4ku]

I increase the number of observations in ChunkingWrapper:

env = ChunkingWrapper(env, obs_horizon=2, act_exec_horizon=None)

And have an error:

Traceback (most recent call last):                               
  File "/ros_ws/src/universal_serl/examples/pick_real/train_reward_classifier.py", line 169, in <module>                          
    app.run(main)                                                
  File "/usr/local/lib/python3.10/dist-packages/absl/app.py", line 308, in run                                                    
    _run_main(main, args)                                        
  File "/usr/local/lib/python3.10/dist-packages/absl/app.py", line 254, in _run_main                                              
    sys.exit(main(argv))                                         
  File "/ros_ws/src/universal_serl/examples/pick_real/train_reward_classifier.py", line 101, in main                              
    classifier = create_classifier(key, sample["next_observations"], image_keys)                                                  
  File "/ros_ws/src/serl/serl_launcher/serl_launcher/networks/reward_classifier.py", line 59, in create_classifier                
    params = classifier_def.init(key, sample)["params"]                                                                           
  File "/ros_ws/src/serl/serl_launcher/serl_launcher/networks/reward_classifier.py", line 22, in __call__                         
    x = self.encoder_def(x, train=train)                         
  File "/ros_ws/src/serl/serl_launcher/serl_launcher/common/encoding.py", line 46, in __call__                                    
    image = self.encoder[image_key](image, train=train, encode=not is_encoded)                                                    
  File "/ros_ws/src/serl/serl_launcher/serl_launcher/vision/resnet_v1.py", line 341, in __call__                                  
    x = self.pretrained_encoder(x, train=train)                  
  File "/ros_ws/src/serl/serl_launcher/serl_launcher/vision/resnet_v1.py", line 223, in __call__                                  
    x = (observations.astype(jnp.float32) / 255.0 - mean) / std                                                                   
  File "/usr/local/lib/python3.10/dist-packages/jax/_src/numpy/array_methods.py", line 265, in deferring_binary_op                
    return binary_op(*args)                                      
  File "/usr/local/lib/python3.10/dist-packages/jax/_src/numpy/ufuncs.py", line 86, in <lambda>                                   
    fn = lambda x1, x2, /: lax_fn(*promote_args(numpy_fn.__name__, x1, x2))                                                       
  File "/usr/local/lib/python3.10/dist-packages/jax/_src/numpy/util.py", line 381, in promote_args                                
    return promote_shapes(fun_name, *promote_dtypes(*args))                                                                       
  File "/usr/local/lib/python3.10/dist-packages/jax/_src/numpy/util.py", line 250, in promote_shapes                              
    result_rank = len(lax.broadcast_shapes(*shapes))                                                                              
ValueError: Incompatible shapes for broadcasting: shapes=[(256, 128, 128, 6), (3,)]                                               

[youliangtan]

Hi @4ku, currently the example is only tested with a history obs of 1, and not yet support stacking of multiple history observations. Technically it can be done easily with the serl_launcher library.

Multiple places of the example codes require modification to make this possible (e.g. init replay buffer, init of agent, etc...). This would something that we can add in a future PR, but feel free to open a PR for this solution too! Thanks

[4ku]

Also have this error, when trying to launch with sim:

Traceback (most recent call last):
  File "/home/ivan/Desktop/RL/serl/examples/async_drq_sim/async_drq_sim.py", line 417, in <module>
    app.run(main)
  File "/home/ivan/.local/lib/python3.10/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/ivan/.local/lib/python3.10/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/ivan/Desktop/RL/serl/examples/async_drq_sim/async_drq_sim.py", line 371, in main
    demo_buffer = MemoryEfficientReplayBufferDataStore(
TypeError: MemoryEfficientReplayBufferDataStore.__init__() got an unexpected keyword argument 'preload_rlds_path'

[youliangtan]

Thanks @4ku for raising the regression error. I have fixed it in https://github.com/rail-berkeley/serl/pull/57

[4ku]

Thank you