RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.28k stars 1.26k forks source link

Model visualizer on dmd-and-more files #20725

Open jwnimmer-tri opened 9 months ago

jwnimmer-tri commented 9 months ago

Is your feature request related to a problem? Please describe.

It's not uncommon to nest a "Drake model directives YAML" tree (aka "dmd.yaml" file) as a sub-tree in a larger file. For example, in the scenario file hardware_sim/example_scenarios.yaml we have this structure:

# This demo simulation shows an IIWA arm with an attached WSG gripper.
Demo:
  directives:
  - add_model:
      name: amazon_table
      file: package://drake/examples/manipulation_station/models/amazon_table_simplified.sdf
  - add_weld:
      parent: world
      child: amazon_table::amazon_table
  - ...
  lcm_buses:
    ...
  cameras:
    ...
...

If a user wants to see what those directives look like, they currently have three options:

(1) Run the simulation and immediately pause it (e.g., by setting a max duration of zero, or epsilon). Then use a visualizer like Meshcat or Meldis to pan & zoom on the scene, check the collisions, etc. One downside: Sometimes it can be tricky to pause it at a useful moment, requiring several iterations to find a good time. This can be mitigated if the simulation saves a meshcat animation, in which case the user would have a time scrubber knob. Another downside: Cannot easily posture the scene. The postures all come from the initial conditions + dynamics -- if the user wants to "see" what it looks like with the objects arranged differently, they need to edit the scenario and re-start it. If they accidentally posture the objects into collision, things explode.

(2) Copy + paste the directives: content into a temp file, and then run model_visualizer to explore it. The downside here is that it's messy / inconvenient to keep up with copy + paste, and probably error-prone.

(3) The user can choose to put a one line "add_directives" in their scenario file, so that the dmd content is already in a standalone file, and then can already model_visualize that dmd file. This has the downside of bloating the number of files on disk, and might introduce other workflow constraints (e.g., if the scenario is auto-generated).

Describe the solution you'd like

Ideally, there would be some way to tell model_visualizer to load a YAML file, navigate to a specific sub-tree within that document, and then visualize that sub-tree as if it were a *.dmd.yaml file. Effectively, this would automate option (2) to be faster and less error-prone.

Describe alternatives you've considered

There might be a way to semi-automate this, without changing model_visualizer, something like (inventing a syntax) jq foo.yaml --keep-only //Demo/directives > temp.dmd.yaml && model_visualizer temp.dmd.yaml.

Additional context

Scenario authoring is an ever-more-important use case of Drake. Big picture, there are other ways to make it easier (e.g., create a GUI) but insofar as we still have users writing raw YAML files by hand, it would help to make it easier to locally iterate on those files quickly.

jwnimmer-tri commented 9 months ago

As it turns out, option (2) can be automated pretty well, with off-the-shelf tools.

This works:

(echo "directives:"; yq .Demo.directives examples/hardware_sim/example_scenarios.yaml) > temp.dmd.yaml && \
  bazel-bin/tools/model_visualizer_private temp.dmd.yaml

We could imagine having a --yq flag built-in for slightly nicer sugar, and we'd call import yq as a conditional import, e.g.

bazel run //tools:model_visualizer -- examples/hardware_sim/example_scenarios.yaml --yq=.Demo.directives