Is it possible to read the color value of a picked point?

[nvaytet]

Hi,

I am trying to add a hover picker on a Points geometry that would allow me to read the color value of the picked point (or even better the index of the picked point in the position array).

Inspired from the Picker example (https://github.com/jupyter-widgets/pythreejs/blob/master/examples/Picker.ipynb), I manage to read the position of the points over which I am hovering. How could I read the color value without having to perform a search for the point that is closest to the current cursor position?

Code example:

import pythreejs as p3
import numpy as np
import ipywidgets as widgets
from IPython.display import display

geometry = p3.BufferGeometry(
    attributes={
        'position':
        p3.BufferAttribute(array=10.0*(np.random.random([100, 3]) - 0.5)),
        'color':
        p3.BufferAttribute(array=np.random.random([100, 4]))
    })
material = p3.PointsMaterial(vertexColors='VertexColors',
                                         transparent=True)
points = p3.Points(geometry=geometry,
                   material=material)

view_width = 600
view_height = 400
camera = p3.PerspectiveCamera(position=[20, 0, 0], aspect=view_width/view_height)
scene = p3.Scene(children=[points, camera], background="#DDDDDD")
controller = p3.OrbitControls(controlling=camera)
renderer = p3.Renderer(camera=camera, scene=scene, controls=[controller],
                    width=view_width, height=view_height)

# Picker object
hover_picker = p3.Picker(controlling=points, event='mousemove')
renderer.controls = renderer.controls + [hover_picker]

# A sphere for representing the current point on the surface
hover_point = p3.Mesh(geometry=p3.SphereGeometry(radius=0.5),
                   material=p3.MeshBasicMaterial(color='hotpink'))
scene.add(hover_point)

# Have sphere follow picker point:
widgets.jslink((hover_point, 'position'), (hover_picker, 'point'))

coord_label = widgets.HTML()  # A label for showing hover picker coordinates

def on_hover_change(change):
    coord_label.value = 'Pink point at (%.3f, %.3f, %.3f)' % tuple(change['new'])

on_hover_change({'new': hover_point.position})
hover_picker.observe(on_hover_change, names=['point'])

# display label and renderer
display(coord_label)
display(renderer)

Many thanks for any help!

[vidartf]

Sorry, no, it isn't possible to get the color. But the index in the position array should probably be possible. This is the information currently returned:

https://github.com/jupyter-widgets/pythreejs/blob/4c40956bb0541335273cf5a352498ac1c91bc8b4/js/src/controls/Picker.js#L128-L136

[nvaytet]

Getting the index would be super. I would be more than happy to create a pull request with changes but would need some pointers as to where I could add this. I am guessing it may be in the object returned by raycaster.intersectObject(root, true)? https://github.com/jupyter-widgets/pythreejs/blob/4c40956bb0541335273cf5a352498ac1c91bc8b4/js/src/controls/Picker.js#L108

[vidartf]

I think you should already be able to get the index (via face/faceIndex I think?). I'm not sure if the problem will be any easier on the JS side.

[nvaytet]

I had tried looking at the face and faceIndex, but for a Points object they seem to always be None or zero. They don't change when I move the mouse over another point in the cloud.

I have indeed tried to replace my Points with a Mesh that uses a BoxGeometry and the faceIndex then seems to be updating with something sensible. Can it be that there are no faces in a Points object?

[nvaytet]

Or maybe it's something to do with the fact that I am using a BufferGeometry? https://github.com/mrdoob/three.js/issues/5164

[vidartf]

I think this is solved by https://github.com/jupyter-widgets/pythreejs/pull/379 . Going to close this for now (expect a release soon, and ping me if not).