Display scalars of different scales separately

[wbrickner]

Describe the annoyance

Logging two scalars, one small and one that becomes very large. They belong to the same plot by default, and only the large value is visible, the small value being squashed to nothing along the X axis.

To Reproduce Steps to reproduce the behavior:

1. Log a big and small valued function.
2. mfw: where's the small function?

Expected behavior

I want Rerun viewer to recognize that the values are wildly different ranges, and separate all scalar time-series out into groups of similar range. If I had 10 scalars of different ranges, perhaps I would end up with 3 plots, for example. Rerun must already compute range of the data, and we can group series by range being within 2-5x one another but not more.

Your goals

Make visualizing the internal state of my software actually frictionless, which rerun takes major steps toward. In my perfect world, where rerun is perfect, I would have to do zero configuration changes to get a good look at the internal state I am trying to visualize. It would be smart enough to solve all "uglies" and "invisibles".

Screenshots

I can edit later with screenshots if you want, but not really necessary. Don't have any on my mobile.

Desktop (please complete the following information):

• OS: Platform independent

Additional context

[nikolausWest]

Hey @wbrickner, thanks for this issue! Really appreciate it! This feels like something we should definitely do. I've added it to our backlog but I can't yet commit to exactly when it will happen.

[emilk]

We should make sure that there is easy way of indicating that the two logged scalars should show up in different plots, ideally already in the logging SDK, otherwise it should be super-simple to split that up in the UI.

[wbrickner]

My opinion, for whatever it's worth, is that there's already a lot of configuration work I needed to do to be translating tensors etc from the Rust API, and I ended up having to wrap the re_sdk stuff in a function to then scatter one liners all around. I think the brightest future would be to make the viewer have really good heuristics ("just works") with optional configuration at log time (or better, at sdk init time) to force specific layouts. Separating values so their shape can be seen simultaneously is a good balance, which the streaming viewer can also do (data get separated once their values diverge dramatically, and that view arrangement is maintained for all times prior).

As an aside, the experience of viewing data once it's all hooked up is generally fantastic, however I found myself frustrated by the fact that you can't really modify the way something is displayed in the editor, and the defaults were not always reasonable. For example, many separate 1D tensors would display as overlapping bar graphs, but i should be able to change them to display as heat maps separately like 2D tensors.

You all are doing really innovative work & I am enjoying using rerun for debugging optimization algorithms (helped me identify like 10 bugs once I could see what was going on!)

[nikolausWest]

@wbrickner

that there's already a lot of configuration work I needed to do to be translating tensors etc from the Rust API, and I ended up having to wrap the re_sdk stuff in a function to then scatter one liners all around.

Could you expand a bit on this? Out goal is for this to be as easy as possible to start with. What was the most annoying / painful? How could we make the api more ergonomic?

I think the brightest future would be to make the viewer have really good heuristics ("just works") with optional configuration at log time (or better, at sdk init time) to force specific layouts.

We totally agree with this vision (with the addition that you should also be able to configure from the UI). We're doing a bunch of supporting work to enable configuration from the SDK right now. Heuristics are always difficult, but important. As we have some big changes underway for how configurations get set we're a bit hesitant at adding in complex heuristics that might make refactoring more difficult. That said, if you can write a heuristic that is relatively isolated that you think does the job, we're definitely open to a PR!

helped me identify like 10 bugs once I could see what was going on!

This warms my heart ❤️

[wbrickner]

Warning: laundry list of armchair complaints and ideas about your beautiful project :) you all seem to really enjoy user perspectives so these are just my thoughts about how you could improve.

wrt boilerplate, a lot of it is not a fixable problem, as it is somewhat irreducible complexity given the flexible ways things can be configured

• needing to setup the global_session and then import that function everywhere
• needing to manually translate arrayfire-rust Array<F> buffers into Rerun buffers. This could be done automatically for me by rerun, but only for a select number of crates like nalgebra, arrayfire, glam, etc (and maybe some blanket implementations of things that are iterable). I am assuming this is mostly how things work for Python users.
• Btw, I thought for a while that there was a bug in my own unsafe buffer shenanigans because, to my surprise, rerun tensors use the less common row-major storage format, which I wasn't expecting.
• I don't think I ever managed to do the translation the right way. I can see my tensors, but I always get warnings when I hover in the viewer. Are 4D tensors well supported? Mine are always [x, y, 1, 1]

The ideal situation from my perspective would be something extremely terse and simple, like a macro-rules expansion:

use re_sdk::*;

// ...deeply nested somewhere...
rerun!(something_loggable);

// or maybe instead:
rerun!(tensor = my_image);
rerun!(scalar = quantity);
rerun!(component); // implements `Component` or `Into<Component>`, so it works

and that macro would just "figure it out", and a connection will be made automatically if it hasn't been configured manually yet, and RERUN env var is set. I am writing libraries that integrate rerun, and I want to give the end user the option to connect a rerun viewer, but not force them to pay any cost (have rerun buffer gigabytes of tensors if no viewer is connected). This can be solved on my end by lazily initializing the global session upon first log attempt, and logging only if an option is set, but it might be better if this use case were better considered in the library.

Finally, on my machine, adding the rerun crate is a big deal. Laptop may begin to levitate, 5-10min of compilation (on debug mode). I don't have the best machine but it is abnormally intense. All I need is the ability to connect and send messages, so I ended up figuring out re_sdk was sufficient, which is pretty lightweight. I think bundling the entire viewer + additional features by default is not the right decision, and people who want to access rerun logs ought to just have the cli installed.

Happy to hear that things are in the works wrt view-time rendering options!

Excited to continue using rerun for my own projects, it's already very useful even with the pain points, and I can only imagine how insanely cool you very smart people will end up making it!

[nikolausWest]

Thank you @wbrickner! A bunch of things here that we should definitely be able to improve upon a lot!

If you don't mind me asking, what libraries are you integrating Rerun into right now?

[wbrickner]

To make sure I answer your question:

Libraries I am authoring

Two libraries, one is an implementation of CMAES, an optimization algorithm that uses no gradients. The other is egirl, an end to end gradient-less machine learning toolkit that uses cmaes internally. May not ever release them depending on how well they work in practice.

Libraries I am using

• arrayfire-rust, a portable GPU lib
• nalgebra, the rust linalg crate
• nalgebra-lapack, bindings to intel-mkl etc.

[nikolausWest]

Thanks @wbrickner!

[wbrickner]

oh, one other thing I've been struggling with!

I want to be able to watch live executions on my laptop, that are running on my laptop, or that will eventually be running on a remote machine.

Is it somehow possible to make this work automagically? It works okay if the machines are on the same LAN (pass an arg for the rerun viewer address) but if not I need a tunnel or a port forward config to make it work.

Maybe rerun could provide a paid service that acts as TCP tunnel / rr log retention someday?

[nikolausWest]

Making it alot easier to set up connections like that is definitely on our minds for a commercial product. I've had that need before a bunch but you are the first person that brings it up in the context of Rerun.

[wbrickner]

Being able to zoom the axes of a 2D view independently would be a huge QOL boost. I have all this data now & I can't really understand it at any zoom scale bc it's too tall.

[emilk]

You can do non-uniform zoom by dragging with the right mouse button (though I notice that is undocumented 😬).