Getting relevant data from Layer

jesusbotella commented 4 years ago

An important part of what consists of our Web SDK is to let users access the data behind the visualization they are watching on their computers' screen, whether it is the features rendered on screen, the extent of the features of the layer or the number of rows of the whole dataset.

So for that matter, new functions have to be added to our Layer specification. Replicating the ones that we had in CARTO-VL, we thought of adding those ones:

Layer.getViewportFeatures()

This method will return an array containing GeoJSON objects for the features that are visible within the current viewport.

Proposed usage:

const layer = new carto.viz.Layer('dataset');
layer.addTo(deckInstance);

// Get current viewport features
(await?) layer.getViewportFeatures();

Depending on whether we do it with an SQL query or filtering features within a layer, we would need to add an await keyword. Both approaches have upsides and downsides.

Filtering the rendered feature data from our layer This approach is based on getting features from MVTLayer, filter them by the bounds of the current viewport and returning them in our function. It is completely synchronous, so no need to await for our results, but filtering happens on client. But given that our coordinates are local coordinates we would need to convert them into WGS84 coordinates before coordinate comparison
Requesting rows within viewport to SQL API This approach is much easier for us given that SQL API will perform the filtering and then we will return the rows' data in our function. The downside is that the request can take too much time depending on the data that is matched, and we will need to add an await keyword to our execution statement.

Layer.getExtent()

This method will return the bounds of our layer in the map, the coordinates of the rectangular area that encloses all layer features.

Proposed usage:

const layer = new carto.viz.Layer('dataset');
layer.addTo(deckInstance);

// Get layer extent
await layer.getExtent();

Return value:

[
 { longitude: -180, latitude:  90 },
 { longitude:  180, latitude:  90 },
 { longitude:  180, latitude: -90 },
 { longitude: -180, latitude: -90 },
]

To get layer's extent, we would need to request it to SQL API, so there's no discussion for this method's approach.

Layer.getNumberOfRows()

I think that was @VictorVelarde who added this method to Clubhouse's task, but I find it kind of strange to ask our layer for the number of rows. It is probably a better fit to have a .getDataset() or .getSource() method, and then request the number of rows to that object. What do you think?

Proposed usage:

const layer = new carto.viz.Layer('dataset');
layer.addTo(deckInstance);

// Get layer extent
await layer.getDataset().getNumberOfRows();
await layer.getSource().getNumberOfRows();

VictorVelarde commented 4 years ago

Hi @jesusbotella thx for your thoughts. Some comments on them.

Indeed, all of them look to me like methods belonging to an internal 'Source' object. The Layer methods would be probably just a facade on them, and a way to get some params for the queries.

viewportFeatures

Layer.getViewportFeatures()looks like a nice method name, that could be handled internally by the source, with the help of layer.getMap().getCurrentExtent() (not real names)

I would start with an await interface and the SQL API, as the first & quick approach. That could be probably enough for smaller datasets. From there, we can start exploring other options, like client-side filtering (that filtering would probably belong to MVTLayer, thus a deck.gl contribution). If we keep the SQL API approach, we should consider pagination as well.

extent

Layer.getExtentlooks nice too, although I would probably change the returned value, to something more explicit, like:

{
  lonMin: -180,
  latMin: -90,
  lonMax: 180,
  latMax: 90
}

or even the common xmin, ymin, xmax, ymax order [-180, -90, 180, 90]

xtra (number of rows...)

It's true, I added it as an example, and it should probably be developed together with some other 'basic stats' info related to the source (https://app.clubhouse.io/cartoteam/story/63820/data-basic-field-stats). So IMO we can postpone it

VictorVelarde commented 4 years ago

I'm gonna close this issue. We should better keep this "open discussions" in Clubhouse and leave GH mostly for PRs

CartoDB / toolkit