search

vega / vega-transforms

Data processing transforms for Vega dataflows.
BSD 3-Clause "New" or "Revised" License
2 stars 6 forks source link

Can not find way to transform a datasource with a query based lookup value from other datasource #4

Closed dimikhan closed 6 years ago

dimikhan commented 6 years ago

I want to lookup the matching record in my secondary datasource with two clauses and one of that clause will be with a range of value. In normal sense it will be like a query to the secondary datasource - find record from data2 where data2.step_id = data1.step_id-1 and data2.@timestamp < data1.@timestamp

It seems to me that lookup transformation does not support two clauses and also it looks for exact value, not within a matching value range. Is there any existing transformation available to meet the requirement? If not can there be an upgrade to existing lookup transformation to meet this requirement?

Note: I tried a filter before lookup. But I can not filter with timestamp data. And more over I can not run filter on secondary datasource. If I use it in my primary datasource, primary datasource itself will be modified. Which I dont want.

dimikhan commented 6 years ago

Solution provided by Roy I - https://groups.google.com/forum/#!topic/vega-js/Q2XrYMOVluw