rerun-io / rerun

Visualize streams of multimodal data. Free, fast, easy to use, and simple to integrate. Built in Rust.
https://rerun.io/
Apache License 2.0
6.52k stars 327 forks source link

Annotation context doesn't correctly project when projecting 3d keypoints into 2d image #2359

Closed pablovela5620 closed 1 year ago

pablovela5620 commented 1 year ago

Describe the bug

https://github.com/rerun-io/rerun/assets/25287427/02c51a77-7bba-4120-9468-a9c3d6e604f6

Currently when projecting a 3d entity into 2d, the annotation context doesn't currently get shown (as in the video above) in 0.6.0

To Reproduce

Steps to reproduce the behavior:

  1. Log an annotation context for a 3d keypoint entity (such as a set of hand or body landmarks)
  2. log said 3d key points in the world space and cameras via pinhole/rigid3d
  3. Go to corresponding image and add entity to view
  4. 2d keypoints are projected fine (in this case they look wrong only because this is being projected as a pinhole camera with fisheye distortion parameters). But annotation context (in this case the links between the keypoints) does not correctly project

Expected behavior

The links between the keypoints should also correctly show in the 2d image (as they do in 3d)

Desktop (please complete the following information):

Rerun version

rerun_py 0.6.0 [rustc 1.69.0 (84c898d65 2023-04-16), LLVM 15.0.7] aarch64-apple-darwin release-0.6 643dea9, built 2023-05-25T20:38:09Z

Wumpf commented 1 year ago

Fairly sure this is fixed by #2407, but someone would need to confirm :)

pablovela5620 commented 1 year ago

Fairly sure this is fixed by #2407, but someone would need to confirm :)

Looks like this was fixed when I upgraded to latest prerelease, but something unrelated popped up when I log Monochrome images. They are much much dimmer now

https://github.com/rerun-io/rerun/assets/25287427/7b55fb8d-c524-436b-bc26-b37ac79d50f8

pablovela5620 commented 1 year ago

Both of these are fixed for me on 0.7.0! Nice job ya'll

Wumpf commented 1 year ago

awesome, thank you for following up!