rerun-io / rerun

Visualize streams of multimodal data. Free, fast, easy to use, and simple to integrate. Built in Rust.
https://rerun.io/
Apache License 2.0
6.68k stars 338 forks source link

Web Socket Client Memory Limit #8173

Open edwardstevenw opened 6 days ago

edwardstevenw commented 6 days ago

Describe the bug

I am running a rerun web-server which logs data over HTTP, and the client side opens a web browser to view the data. The problem I encountered is the memory of my client is piling up due to the memory limit in the client side rerun browser is set to 2.3GB. I am logging high frequency live data, so it piles up the memory used by the web browser very quickly and freezes the system. Is there a way to limit the memory in rerun web browser.

To Reproduce Steps to reproduce the behavior:

  1. Run this code:
    rerun --serve-web --server-memory-limit 100MB --memory-limit 100MB
    [2024-11-19T08:21:53Z INFO  re_sdk_comms::server] Hosting a SDK server over TCP at 0.0.0.0:9876. Connect with the Rerun logging SDK.
    [2024-11-19T08:21:53Z INFO  re_ws_comms::server] Hosting a WebSocket server on ws://localhost:9877. You can connect to this with a native viewer (`rerun ws://localhost:9877`) or the web viewer (with `?url=ws://localhost:9877`).
    [2024-11-19T08:21:53Z INFO  re_sdk::web_viewer] Hosting a web-viewer at http://localhost:9090?url=ws://localhost:9877
  2. Open the localhost web socket client and open the toggle memory panel to see the memory limit image
  3. On the bottom left, memory limit is 2.3GB

Expected behavior

Running rerun viewer natively does what is expected, the memory limit can be set and will drop old data when memory limit is reached.

rerun --memory-limit 100MB
[2024-11-19T08:29:25Z INFO  re_sdk_comms::server] Hosting a SDK server over TCP at 0.0.0.0:9876. Connect with the Rerun logging SDK.

image On the bottom left, memory limit is ~100MB which is what was expected.

Desktop (please complete the following information):

Rerun version

rerun-cli 0.20.0 (default map_view nasm native_viewer release web_viewer) [rustc 1.79.0 (129f3b996 2024-06-10), LLVM 18.1.7] x86_64-unknown-linux-gnu release-0.20.0 2bed90b, built 2024-11-14T15:43:35Z Video features: av1 default ffmpeg nasm serde

teh-cmc commented 6 days ago

There is no way to set the memory limit of the web viewer as of today (it always defaults to 2GiB -- wasm32's memory limit).

Is there any reason your client cannot use the native viewer btw? The native viewer can connect to a Rerun WebSocket server just the web viewer does (with orders of magnitude better performance). E.g. rerun ws://1.2.3.4:9877.

Related:

edwardstevenw commented 6 days ago

The client I am using here is a tablet device, so I am not able to install rerun native in there, therefore I was using the rerun web browser to view the data from the server.