search

tensorflow / tensorboard

TensorFlow's Visualization Toolkit
Apache License 2.0
6.68k stars 1.65k forks source link

Visualize Feature Embeddings for 3D Models #4365

Open albertotono opened 3 years ago

albertotono commented 3 years ago

It would be great to be able to visualize 3d models in the latent space with Tensorboard, navigating the space instead of having picture see the 3d models.

Something like this https://twitter.com/marian42_/status/1188969971898048512?lang=en

stephanwlee commented 3 years ago

Thanks for the feedback. Have you seen our embedding projector? (Demo at https://projector.tensorflow.org/). It surely is not as cool looking as the gif in the tweet but I believe it is what you are looking for.

ICYMI, here is the tutorial for the projector plugin: https://www.tensorflow.org/tensorboard/tensorboard_projector_plugin

albertotono commented 3 years ago

Hi @stephanwlee, thank you so much for the quick reply , yes I was looking at that, but I haven't seen a way to display 3D shapes. Is it possible?

Or as workaround, I can take a picture of the shape and use that to display in the t-SNE.

stephanwlee commented 3 years ago

I believe it is possible to show 2d image!

If you use the left nav and select "Mnist with Image", you will see an example it showing MNIST image.

image

albertotono commented 3 years ago

Yes, I know that it is possible to display images directly, is you are working with images,. But it would be nice to have directly 3d models as a future feature, something similar to this paper and their work here.

Screenshot from 2020-11-23 08-42-42

So when I click on the image of the model, it connect to the 3d shape. Probably computationally speaking will be to expensive. Any thoughts?

stephanwlee commented 3 years ago

Or as workaround, I can take a picture of the shape and use that to display in the t-SNE.

Got it, I was not sure whether you were acknowledging the possibility in current tool.

While I captured this as a feature request, I cannot imagine this being compelling for large number of our users and make it to our priority list.