kylebarron / deck.gl-raster

deck.gl layers and WebGL modules for client-side satellite imagery analysis
https://kylebarron.dev/deck.gl-raster/
MIT License
85 stars 10 forks source link

Layer Picking #32

Open kylebarron opened 4 years ago

kylebarron commented 4 years ago

Trying to pick from rendered layers gives color: null and picked: false.

kylebarron commented 4 years ago

Note that this is "fixed" by just setting pickable: true on the layer. However this still gives object: undefined in the picked result because deck.gl is focused on vector picking. So when you pick it gives you the tile, not a pixel within the tile.

So you'll probably need to dig into deck.gl's picking to figure out how to pick the pixel value itself.

kylebarron commented 4 years ago

Debug using drawPickingColors: https://github.com/visgl/deck.gl/blob/8ed7b8e535c7867724bd9ba9a0f35f89bfbb325d/modules/core/src/lib/deck.js#L89

When you call deck.pickObject, it delegates to this.deckPicker, which is defined in deck-picker.js. That calls DeckPicker.pickObject, which calls DeckPicker. _pickClosestObject.

It looks like then DeckPicker._drawAndSample is called

  // returns pickedColor or null if no pickable layers found.

which internally calls this.pickLayersPass.render. That's defined in pick-layers-pass.js.

At a low level, you should read the Luma picking dev docs and the luma picking module code.

kylebarron commented 4 years ago

If you don't supply the picking module in the layer, then "color" picking will be correct. I.e. it will pick the actual color rendered to the screen.

That said, the color attribute is always a Uint8ClampedArray unless pickZ is true, in which case it's a Float32Array then? The easiest way for pickZ to be true is to have unproject3D to be true

kylebarron commented 4 years ago

Therefore, the best next try is to create a custom picking module that takes the same uniforms, and breaks out of computation at that point (hence stopping modification of color) if picking is active. Something like

if (picking_uActive) {
  return color;
}

top-level in main? And then would return e.g. the NDVI computed value