search

ARISE-Initiative / robomimic

robomimic: A Modular Framework for Robot Learning from Demonstration
MIT License
592 stars 181 forks source link

Question about Hidden State Reinitialization in LSTM for Behavior Cloning #164

Closed pengzhi1998 closed 2 months ago

pengzhi1998 commented 3 months ago

Dear Authors,

Thank you again for this great repo!!! I have a question about the rollout/inference using LSTM with Robomimic. I'm confused about the implementation in bc.py. When config.algo.rnn.horizon is set to 10 (the default value in the config files), the hidden state is reinitialized to zeros during inference every 10 time steps.

If I understand correctly, this means that at timestep 10, the hidden state is reset, and at timestep 11, it would lost information from all previous timesteps (even information from timestep 10) due to the reset. This behavior seems a bit counterintuitive to me. Could you please clarify this? Or perhaps I misunderstood the process?

Thank you in advance! Look forward to your reply!

Best regards, Pengzhi

amandlek commented 2 months ago

Your understanding is correct! We found this to work well empirically. Another option is to framestack the observations, and completely unroll on every single timestep - this is actually what BC-Transformer does. However, we found our current approach to work comparably or better, so that's what we have kept as the main implementation for BC-RNN

pengzhi1998 commented 2 months ago

Got it! Thank you so much for your response!! But may I ask whether BC-Transformer's performance is better than BC-RNN in most of the tasks?

amandlek commented 2 months ago

We saw that it was usually around the same performance on most tasks.

pengzhi1998 commented 2 months ago

Got it. Thank you!