search

pleonard212 / pix-plot

A WebGL viewer for UMAP or TSNE-clustered images
MIT License
597 stars 139 forks source link

Points should remain orthogonal to camera in 3D space #231

Open harveyng opened 3 years ago

harveyng commented 3 years ago

Hi, I am trying to display 3d umap on pixplot, everything work fine except for when i turn on Rotation control. When i rotate around the images, the images not rotate with the camera it so its not facing to the camera. What else should i do to make it work probably? Thanks

controls.noRotate = false;

duhaime commented 3 years ago

I'm glad you're thinking through 3D!

You may be expecting behavior like that of GL_Point primitives, which always face the camera. In our case we have quads so they need to be rotated in the shaders to always face the camera...

We just pushed a branch flight-controls that may be of interest. If you install that branch you can just use:

pixplot --images "my/*.jpg" --n_components 3 and you'll get a 3D umap.

If you try that branch, you can add ?fly=true to the url and you'll be able to use fly controls (WASD keys) to fly through the space:

output

harveyng commented 3 years ago

Thanks @duhaime , I have just tried the flight-controls and it's great.

Using umap3d make rasterfairy unavailable but i think since pixplot have started to support RAPIDS, we can generate both umap3d and umap2d without taking too much time so we still have the sweet rasterfairy visualisation with 3d umap.

Regarding the all angle view, i am quite new to three.js, can you explain what's the advantage of quads over using GL_Point? For visualisation purposes, i think it looks better so why dont we make it as a default for current pixplot version? Or if you have already have it, can you share a example, that would help a lot.

LaurensVoerman commented 2 years ago

I've created a pull request (#251) creating the quad in camera space instead of world space. This will keep the images alligned to the screen. @harveyng: on your question regarding quads vs GL_Point: the points have implementation dependend restrictions on their size (GL_POINT_SIZE_RANGE) making them very difficult to work with in a portable manner. Laurens.