RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.16k stars 1.24k forks source link

Add smoothing to Meshat realtime rate display #21512

Open jwnimmer-tri opened 3 weeks ago

jwnimmer-tri commented 3 weeks ago

Is your feature request related to a problem? Please describe.

In #16981 we added a nice realtime rate ("RTR") chart to Meshcat. A simulation (or really anything with a sense of time) can post its timing information into Meshcat, and then the information is sent to the display to show in a nice animated chart.

However, the chart is often pretty difficult to read. For example when a Simulator has many systems inside, with different update rates, the computational cost will be larger at the "beats" where the updates are nearby, and seem to speed by quickly when no updates are happening. This is especially acute with very slow computations, e.g., simulated camera rendering. The chart also shows a "NN%" text readout, which flickers wildly with the beats, nearly impossible to read even the first sig fig.

Describe the solution you'd like

Somehow the information in the chart should be smoothed, so that it's easier to eyeball -- both in chart form and the "NN%" form.

Roughly the time constant we want to smooth across is on the order of 10 Hz. Most periodic systems in our simulations go at least as fast as 5 Hz, more typically 10 Hz, 16 Hz, 32 Hz, etc. We don't want smoothing 0.1 Hz -- that would not respond quickly enough. Consider for example when the robot grabs something, we want the user to see (somewhat quickly) how the RTR changes when the grasp moves from open to closed. Let's say with 0.25seconds the chart should reflect the new timing.

I don't know if we want a sliding window filter, or exponention, or whatever. We might need to experiment.

Describe alternatives you've considered

Have the caller do the smoothing, and pass the updates to Meshcat less frequently. This seems like it would be worse, because we would have multiple callers to fix.

Additional context

There is a TODO here: https://github.com/RobotLocomotion/drake/blob/ce2d4ae9bb82e953ac66c1f67dbe2356b6f4c2a2/geometry/meshcat.h#L554

jwnimmer-tri commented 3 weeks ago

From a quick code skim, it's also possible that by default we have three MeshcatVisualizer objects they are all fighting about the rate. We should be sure that when using AddDefaultVisualization, there's only 1 place that controls the rate display.

SeanCurtis-TRI commented 3 weeks ago

Quick design proposal after initially poking around:

Current API

We have the Meshcat::SetRealtimeRate() API. A user with access to Meshcat instance invokes it, passing its pre-computed rate value. Meshcat then packages it up into a websocket message that is caught and processed by some javascript in Drake's meshcat.html. The page's javascrpt writes the message value to a local value and continues updating the chart based on the cached value indefinitely. This means, when it is no longer connected to the server, the last reported rate fills up the chart.

Design constraints

  1. We're talking about averaging rates, so simply taking a bunch of rate values and averaging them would be highly inappropriate. To average them, we'd have to be able to weigh them (the wall-clock time would serve as the weight).
  2. Any piece of code with access to a Meshcat instance can redundantly call SetRealtimeRate(). Even if it provides appropriate weight values, multiple systems in a diagram could end up reporting the same rate at the same simulation time step. This would implicitly increase the weight for the rate value with duplicates.
  3. We know that the wall clock time of a simulation step can vary widely (even if we're calling SetRealtimeRate() is called with uniform steps in simulation time).

Proposal

Replace SetRealtimeRate() with an alternate API, something like NoteTimeAdvancement(double time) (or some such thing), with the following semantics:

As implemented, the function would do something of the following:

The aggregator would be akin to a moving average, except:

I'm currently favoring a simpler implementation that is more tailored to Meshcat's use: