senx / warp10-platform

The Most Advanced Time Series Platform
https://warp10.io/
Apache License 2.0
383 stars 52 forks source link

Add possibility to have the filtered elements during a FILTER #219

Closed PierreZ closed 6 years ago

PierreZ commented 6 years ago

Here's an example:

NEWGTS "GTS1" RENAME 
{ 'label0' '42' 'env' 'prod' } RELABEL

NEWGTS "GTS2" RENAME 
{ 'label0' '43' } RELABEL

2 ->LIST

[ SWAP [] { 'env' 'prod' } filter.bylabels ] FILTER

At the end of the script, I have the right result, but in this case, having the the filtered elements during FILTER could be interesting.

Is it possible to have (for example) a new type of FILTER framework , which will push back 2 lists in the stack:

hbs commented 6 years ago

Have you tried converting the resulting lists in sets (via ->SET) and use DIFFERENCE on them? Would that provide the behavior you want?

hbs commented 6 years ago

<% DUP 0 GET SWAP FILTER ->SET SWAP ->SET SWAP 2 DUPN SWAP DIFFERENCE SET-> SWAP SET-> ROT DROP %> 'filter' STORE

Then call '@FILTER'

aurrelhebert commented 6 years ago

I found this hint not really readable

aurrelhebert commented 6 years ago

Basically, it should be a macro that you put server side. You can also use the DOC function to document it. You can re-use it in your code using @filter (I would advise you to change the macro name to neg-filter as example)

aurrelhebert commented 6 years ago

Here Pierre, I reworked a bit Mathias proposal to let you have an understanding of what is done:

//
// How to overwrite a WarpScript Filter
//

<%

  //
  // Here you have on the stack the complete FILTER parameter list 
  //

  DUP 

  //
  // keep only the GTS data list
  //

  0 GET

  //
  // Swap the first GTS list with the FILTER parameters list and Apply initial filter
  //

  SWAP FILTER

  //
  // Here you get FILTER list as result
  //

  //
  // Convert it to a SET
  //

  ->SET 

  //
  // Do the same for the GTS data list still on the stack
  //

  SWAP ->SET 

  //
  // Then you apply a SET DIFFERENCE
  //

  SWAP 2 DUPN DIFFERENCE 

  //
  // Convert both result back to a list
  //

  SET-> SWAP SET->

  //
  // Delete initial SET still on stack
  //

  ROT DROP
%>
'neg-filter' STORE

//
// Use case example
//

//
// Create a GTS test list
//

[
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'name' RENAME
]

//
// Apply a new neg filter on it
//

[ SWAP [] 'name' filter.byclass ] @neg-filter

Personally, I would have use some variables to get a code a little bit more readable.

LostInBrittany commented 6 years ago

My take, using variables as @aurrelhebert suggested:

<%

  'params' STORE                     // Here you have on the stack the complete 
                                     // FILTER parameter list 

  $params FILTER                     // Apply the filter

  ->SET  'filtered_gts' STORE        // Convert it to a SET

  $params 0 GET ->SET 'gts' STORE    // Do the same for the initial GTS list

  $filtered_gts $gts DIFFERENCE      // Then apply a SET DIFFERENCE

  SET-> 
  $filtered_gts SET->                // Convert both result back to a list

%>
'neg-filter' STORE

// Use case example

// Create a GTS test list

[
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'name' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'not' RENAME
    NEWGTS 'name' RENAME
]

// Apply a new neg filter on it

[ SWAP [] 'name' filter.byclass ] @neg-filter
PierreZ commented 6 years ago

Thanks for the commented macro guys!

hbs commented 6 years ago

Does this macro provide the requested behavior?

PierreZ commented 6 years ago

Yes it's working, thanks everyone! Closing the issue