search

rerun-io / rerun

Visualize streams of multimodal data. Free, fast, easy to use, and simple to integrate. Built in Rust.
https://rerun.io/
Apache License 2.0
6.67k stars 337 forks source link

Support for more 2D and 3D primitives #1361

Open jleibs opened 1 year ago

jleibs commented 1 year ago

It would be nice to be able to log the common 2D/3D geometry building blocks in order to enable building up simple 3D models without going all the way to fully specified meshes.

Some basics:

Maybe more, but good enough for VRML, good enough for me?

These should all consistently support components for OutOfTreeTransform3D, Color, and Fill.

emilk commented 1 year ago

I think we should add a minimal version of this ASAP - it can be something that just generates meshes during scene construction for now. This is very limiting at the moment, and we need this to test other things like transparency, 2D layering, and styling.

emilk commented 1 year ago

An interesting note: a Rectangle component is useful both in 2D and 3D.

For 2D we support it via specifying its min/max corners, or center and size, and some other options. We will also add support for 2D transforms (https://github.com/rerun-io/rerun/issues/349).

For 3D we may want the same 2D interface plus a 3D transform, treating the 2D rect as a 3D shape with an implicit Z=0.

emilk commented 1 year ago

Rect/Box we have, but needs styling (Separate issue).

Cylinders and capsules I think are very useful (e.g. limbs)

2D circles/discs are very useful. So are spheres. What is the difference with point?

Point radii can be auto and ui-based, but spheres/circles need a real radius. A sphere/circle would squash with non-uniform transforms. Points are using a faster, less accurate, renderer.

jeongseok-meta commented 10 months ago

It would be great to have a way to visualize the grid or ground. This could be achieved through a loggable object or simply as a visual artifact.

Wumpf commented 10 months ago

@jeongseok-meta yeah I think this should be builtin feature of the viewer that doesn't require logging anything (other than maybe logging the settings itself on the space view once code configurable views are there). There's already a separate issue for this:

mont1004 commented 7 months ago

Hi there.

Just curious, has there been any progress or updates on this topic? Would love to know what's going on. I personally hope for support of an ellipsoid representation in physical units, which can be beneficial for expressing the ambiguity in, for example, camera pose estimation.

abey79 commented 7 months ago

@mont1004 Unfortunately the next release will have little new on this front, but this is still something that's high on our priority list. Do you have any particular primitive in mind that you would need most?

mont1004 commented 7 months ago

@abey79 Thank you for the quick reply!

My team has been developing Visual SLAM, and we've been eyeing the rerun as a top candidate for our new visualizer. As I mentioned earlier, we currently represent the covariance of the estimated camera poses with ellipsoids. Having ellipsoids that can be scaled and compared in size with one another would be very helpful, as this capability is crucial for us.

pablovela5620 commented 7 months ago

Want to add like #6012 that adding text to an image, similar to OpenCV would be appreciated, oftentimes I don't want to log something in textlog/textdocument as I would like it to show in the image itself. As a work around I log a point2d or linestrip with 0 radius and add a label. This works, but I can't easily change the size of the font like cv2.drawtext would allow

nikolausWest commented 7 months ago

@pablovela5620: Are you adding the text to a particular place in the image (i.e. tying it to a geometric location), or is the text more of a description of the image (i.e. it could alternatively be below the image for instance)?

pablovela5620 commented 7 months ago

Generally to a geometric location, for example detecting someone's face and placing the text next to or above their detected face. Then telling them to either get closer or farther from the screen or to move their face to a specific location. Below an image could also be helpful (though I think this would already be possible with a textdoc + blueprint?)

nikolausWest commented 5 months ago

Rays and planes as per #2179 would be great

emilk commented 5 months ago

For the 3D primitives, we should have a style option to either render them as solid, as wireframe, or both. The spatial space-view can convert them to a re_renderer mesh and/or lines as needed. Initially we could keep the coarseness of these primitives hard-coded (i.e. how many subdivisions of a sphere).

To keep things simple, I suggest we use the ~Transform3D~ OutOfTreeTransform3D archetype to orient and position the shapes. This means that e.g. a cylinder would always have its base at the origin, and be defined by Cylinder { radius, height }.

Which axis should "height" be along? I doesn't really matter, but once we picked something we should stick with it forever, and also use it for all other shapes (i.e. the axis along which a capsule and cylinder stretches). It's worth spending some time investigating what is the most common choice in other visualization tools. (VRML uses Y as primary axis)

We should still have nice helpers to construct these primitives. For instance, defining a capsule by two points and a radius (a fat line segment) but then logging that as a single component Capsule { radius, length } plus an ~Transform3D~ OutOfTreeTransform3D. ~This would mean an archetype Capsule would also contain archetype.Transform3D in a sense, which may require some more codegen design/thinking.~

For the 3D ellipsoids: I did some preliminary experimentation with these in https://github.com/rerun-io/rerun/pull/4991

teh-cmc commented 5 months ago

We should still have nice helpers to construct these primitives. For instance, defining a capsule by two points and a radius (a fat line segment) but then logging that as a single component Capsule { radius, length } plus an Transform3D. This would mean an archetype Capsule would also contain archetype.Transform3D in a sense, which may require some more codegen design/thinking.

We use the OutOfTreeTransform3D component for this today; it has worked well for Asset3D.

(PS: don't mind my edit on your comment, I inadvertently pressed Edit instead of Quote -.-)

emilk commented 4 months ago

A less important, but very nice, primitive, is the tapered capsule:

image

kpreid commented 4 months ago

Progress update:

More 3D shapes to come!

emilk commented 3 weeks ago

What is the main axis of a cylinder/capsule?

Finding this information online wasn't easy. A lot of libraries/format don't document this at all.

These are the only ones I've managed to confirm (so far):

X

Y

Z

jleibs commented 3 weeks ago

ROS Viz-marker aligns with urdf/xacro spec in using Z for the length/height axis: https://wiki.ros.org/rviz/DisplayTypes/Marker#Cylinder_.28CYLINDER.3D3.29

emilk commented 3 weeks ago

A lot of 3D libraries have a configurable main-axis, that the user can set to any Vec3. I think we should provide the same. That way a user can easily configure where a cylinder points, without having to calculate a quaternion and read our docs about what our default is.

So I suggest we introduce a components.MainAxis3: datatypes.Vec3 that defaults to +Z. A user can still add any transform on top of it (including rotation).