DAIRLab / dair_pll

DAIRLab Physics-based Learning Library
BSD 3-Clause "New" or "Revised" License
10 stars 5 forks source link

Update video generation to use Drake's VideoWriter instead of Meshcat #3

Closed ebianchi closed 1 year ago

ebianchi commented 1 year ago

With some Drake changes, our video rollout generation using Meshcat had broken. These changes switch to the newer Drake VideoWriter. The generated rollout video is created via the same inputs (a DrakeSystem, a pytorch tensor of a trajectory, and a frame rate) and returns the same outputs (a (1, T, 3, H, W) numpy array of an RGB frame per time step, and a true frame rate).

One of the Drake functions used by this fix is an overloaded version of SceneGraph.RemoveRole, functionality which first made it into the Drake nightly release on January 24th, 2023 -- again, as of the time of making this PR, this change requires Drake functionality that is not yet in the stable Drake release.

Some nice-to-haves that are not yet implemented include:

An example of a generated toss video using these updates: output_

ebianchi commented 1 year ago

Update: With the latest commit, the ground plane is also rendered. The ground is slightly transparent to illustrate any penetrating geometries. output_

ebianchi commented 1 year ago

The latest commit addresses all of the requested changes, however with some comments on the VIDEO_FILENAME definition -- see that conversation above.

ebianchi commented 1 year ago

Added a TODO comment and resolved the remaining conversation. Ready for another review or merge when you're ready, @mshalm.