search

StanfordVL / iGibson

A Simulation Environment to train Robots in Large Realistic Interactive Scenes
http://svl.stanford.edu/igibson
MIT License
659 stars 159 forks source link

Renderer stateful w.r.t. viewpoint info #264

Open cgokmen opened 2 years ago

cgokmen commented 2 years ago

Currently, the way of using the renderer is first setting the camera and then calling render.

renderer.set_camera(*viewpoint_info)
renderer.render()

The renderer thus stores its camera state (e.g. the V and P matrices etc) and uses this when rendering. But for one simulation step there's not one camera angle: the renderer camera can get changed multiple times in a single simulator step based on the viewers enabled, the number of robots in the scene, so on and so forth. Thus relying on this global camera state is dangerous and bug-prone.

Instead, we should be able to simply do:

renderer.render(*viewpoint_info)

and render from a given viewpoint.

fxia22 commented 2 years ago

Good point. We can have a camera object to store the viewpoint info, and the render function takes the camera object.

Note that we need to store the current viewpoint and previous viewpoint to render optical flow and scene flow information.

cgokmen commented 2 years ago

@fxia22 In that case our optical flow/scene flow setup could be quite buggy unless some code takes special case to reset the previous viewpoint only when the simulator timestep has changed.

cgokmen commented 2 years ago

(I'm saying this because on every simulation step currently, many things can call the render function)