search

isaac-sim / OmniIsaacGymEnvs

Reinforcement Learning Environments for Omniverse Isaac Gym
Other
835 stars 211 forks source link

Frame stacking for learning from images #171

Open elle-miller opened 3 months ago

elle-miller commented 3 months ago

Hi there,

What is the best way to render and stack multiple image frames e.g. 4 into a single observation?

For the Camera Cartpole task (code), a pytorch_listener (docs) is used to get the images:

# retrieve RGB data from all render products
images = self.pytorch_listener.get_rgb_data()

However I am not sure how to have multiple images returned. I tried lowering the rendering_dt to a fraction of the sim dt but the function still only returns 1 image. If you could point me to some documentation that would be great. Thanks!