rerun-io / rerun

Visualize streams of multimodal data. Free, fast, easy to use, and simple to integrate. Built in Rust.
https://rerun.io/
Apache License 2.0
7.07k stars 354 forks source link

Improve support for lidar data #4409

Open roym899 opened 1 year ago

roym899 commented 1 year ago

Is your feature request related to a problem? Please describe. Lidar data is quite similar to back-projected depth maps, so it would be nice to support distance-based color maps that can be changed from inside the viewer.

Raw lidar data often comes in terms of angles, distances, and per-point timestamps; processed data (projected and potentially motion compensated) typically comes as 3D point clouds in the sensor frame + sensor pose. I believe the latter would be more important to support directly.

Similar ideas apply to, for example, radar data; so maybe it would make sense to name the archetype in a more general way than just LidarScan.

Things I'm unsure of

Describe the solution you'd like Add a LidarScan archetype (or something more general like RangeScan maybe) that takes 3D points and supports different color maps based on the Euclidean distance to the scan origin.

Specifically for lidar, special per-point values that might be nice to support are the return number (integer) and intensity (float).

It would be nice if the color map input could be changed between distance / return number / intensity / x-value / ...

Describe alternatives you've considered It's already possible to log lidar scans as points, but the color map has to be calculated manually and can't be changed from inside the viewer.

Divelix commented 4 months ago

I found link to this issue in lidar example and it uses matplotlib for colormaps. There is colormap python lib that has single dependency on numpy: cmap (no need to install whole matplotlib). Code example to colorize lidar cloud by distance from origin in rainbow colors using cmap:

import rerun as rr
import numpy as np
from cmap import Colormap

def colorize_radial(pc: np.ndarray):
    r_dist = np.linalg.norm(pc, axis=1)
    r_norm = (r_dist - r_dist.min()) / (r_dist.max() - r_dist.min())
    cmap = Colormap("yorick:rainbow")
    return cmap(r_norm)

if __name__ == "__main__":
    rr.init("lidar", spawn=True)
    cloud = np.array(...) # load (n,3) point cloud
    colors = colorize_radial(cloud)
    rr.log("cloud", rr.Points3D(cloud, colors=colors, radii=0.01), static=True)

There are a lot colormaps in cmap that can be used as reference for native support in rerun if needed.