search

mitre / sparklyr.nested

A sparklyr extension for nested data
Apache License 2.0
31 stars 4 forks source link

filter with lambda #17

Open mattpollock opened 6 years ago

mattpollock commented 6 years ago

from https://jira.apache.org/jira/browse/SPARK-23909

SELECT filter(ARRAY [], x -> true); -- []
SELECT filter(ARRAY [5, -6, NULL, 7], x -> x > 0); -- [5, 7]
SELECT filter(ARRAY [5, NULL, 7, NULL], x -> x IS NOT NULL); -- [5, 7]

Don't want to overload filter but it isn't an sdf_ function, strictly speaking. Maybe the prefix lam_ or something for lambda?

df %>%
  lam_filter(fld, x -> x > 10)
mattpollock commented 6 years ago

tracked with #14