Picking API

[vallsv]

Now a plot can contain many displayed items, including data arrays.

On my actual use case, the plot contains at least

• Several tiles
• A heatmap

It would be good to provide a helpers for picking data from the data coords.

To retrieve:

• the data as a specific data coord
• the data index at a specific data coord

The plot already have the full knowledge of the transformations. Which can be very complex for items like tiling. So i would prefer not to compute such thing on my side.

The aim for now is to feed a tooltip.

[axelboc]

I've done some deeper investigating on this in the past few days.

We are currently limited by the fact that we've opted out of React Three Fiber's events system due to a bug with pointer capture (cf. #994 and #995). The onPointerUp event was not always called when clicking on the canvas and releasing the mouse outside of it, leading to the pointer not being released (thus breaking interactions like Pan or SelectToZoom).

It is my hope that this bug is no longer present in the latest version of R3F. So once we upgrade R3F (as part of the React 18) upgrade, we can try removing useCanvasEvents and re-opting-in to R3F's events system instead (with <mesh onPointerMove />).

Assuming the pointer capture bug is resolved, opting in to R3F's events system should make implementing a Picking API much more straightforward:

1. The tooltip component registers a pointermove handler on a mesh spanning the canvas; thanks to R3F's events system, all the events received now include an array of intersected Three.js objects.
2. Components that want to contribute values to the tooltip can somehow (most likely with a zustand store injected via context) register themselves to the tooltip by providing:
   • a reference to their underlying Three.js object;
   • a callback function that accepts data coordinates and returns the value to display in the tooltip.
3. In the pointermove handler, the tooltip component computes the data coordinates with worldToData(evt.unprojectedPoint) and then loops through the intersected Three.js objects; if it finds one that is registered, it calls the corresponding callback function with the data coordinates; all the values returned by the callbacks are kept in an array.
4. The tooltip component calls a render prop with this array so the consumer can decide how to display them.