rerun-io / rerun

Visualize streams of multimodal data. Free, fast, easy to use, and simple to integrate. Built in Rust.
https://rerun.io/
Apache License 2.0
6.5k stars 322 forks source link

Provide an utility to compute partitions from a list of possibly-repeating timestamp #7333

Open abey79 opened 2 months ago

abey79 commented 2 months ago

A possibly recurrent user scenario is to have data as two arrays of the same length:

For the rr.send_columns API, one must prepare a list of non-repeating timestamp, and the matching "partitions", i.e. the size of the groups in the data that corresponds to each non-repeating timestamp. It's sufficiently annoying to do with numpy that this would warrant an helper, such as to make the following code more compact:

points = ... # numpy array of points
times = ... # numpy array of possibly-repeating timestamps

# indices at which `times` changes, excluding 0, including `n`
change_indices = (np.argwhere(times != np.concatenate((times[1:], [np.nan]))).T + 1).reshape(-1)

# non-repeating timestamps
non_repeating_times = times[change_indices - 1]

# partitions (e.g. size of groups corresponding to each 
partitions = np.concatenate(([change_indices[0]], np.diff(change_indices)))
assert np.sum(partitions) == len(times)

# logging

rr.send_columns(
    "/entity/path",
    [rr.TimeSecondsColumn("time", non_repeating_times)],
    [rr.components.Position3DBatch(positions).partition(partitions)]
)
abey79 commented 2 months ago

This example includes an implementation of such a helper: