search

rerun-io / rerun

Visualize streams of multimodal data. Fast, easy to use, and simple to integrate. Built in Rust using egui.
https://rerun.io/
Apache License 2.0
5.66k stars 255 forks source link

Support huge point clouds in the viewer #1136

Open emilk opened 1 year ago

emilk commented 1 year ago

The Rerun Viewer is currently limited to only being able to display ~two million points at a time, and will have pretty terrible frame rate at 1 million points already.

The reason for this is that we upload the point cloud every frame. This is very simple, but obviously very wasteful.

A solution is to detect if the same points are being rendered this frame as the previous, and if so not re-upload them. This is similar to how we already handle tensors.

Wumpf commented 1 year ago

related to

EriKWDev commented 1 year ago

I have been tackling rendering of point clouds for some time and here are some resources I've used to speed up rendering of point clouds:

While a bit specific to Unreal Engine 4, there are some insights in this master thesis project ( https://cgvr.cs.uni-bremen.de/theses/finishedtheses/point_cloud_rendering_in_unreal/thesis_FINAL_WEB.pdf ) about point order

Wumpf commented 1 year ago

We should be a bit more specific with the "huge" part. From what we learned so far most users are more than happy in the region of <8mio points which seems to be in the region where we don't need acceleration structures for rendering.

We need typically 24bytes per point right now (position, radius, color, index - could compress away the index!), so 8mio points would be merely 184Mb. Given that upload speeds can be assumed to be at the very least 16GB/s (that's PCIe 3.0 16x; 5.0 is common and has 64GB/s and integrated GPUs are off the chart there) we get 273Mb/frame for 60fps. So even if we upload ever frame we should still hit at least 30fps at that size if we do everything correctly.

Right now we have a hardcoded limited 4mio; point cloud renderer should be dynamic with this!

abey79 commented 10 months ago

potree does large point clouds:

emilk commented 10 months ago

The current state is around 1.5M points @30 fps on my M1 MacBook Pro.

We are bounded primarily by

A rough roadmap:

teh-cmc commented 10 months ago

potree does large point clouds:

From the same authors, a drastically different approach by accumulating data across frames, removing the need for hierarchical datastructures, LOD, caching, etc: https://github.com/m-schuetz/Skye

emilk commented 9 months ago

examples/python/open_photogrammetry_format/main.py, showing just the 3D view, on --release, averaged over a few frames, MacBook Pro M1