nathanmarz / cascalog

Data processing on Hadoop without the hassle.
Other
1.38k stars 178 forks source link

ClojureFilter receiving no fields in filterCall arguments in Cascalog 2.0 #214

Open dkincaid opened 10 years ago

dkincaid commented 10 years ago

A JCascalog CascalogFilter that we are using isn't getting any fields passed in as part of the filterCall argument.

Here is the filter:

private static class FilterNullContentTypes extends CascalogFilter {
        @Override public boolean isKeep(FlowProcess process, FilterCall filterCall) {
            int i = filterCall.getArguments().getFields().getPos("!content-type");
            String value = filterCall.getArguments().getTuple().getString(i);
            return !StringUtils.hasText(value);
        }
    }

filterCall.getArguments() is returning: "fields: [NONE] tuple: []"

The SubQuery where it is being used looks like this:

        Subquery filterQuery = new Subquery("?practice-id", "!create-date", "!content-type", "?patientid",
                "?source", "?create_timestamp")
                .predicate(outerJoinQuery, "?practice-id", "!create-date", "!content-type", "?patientid", "?source",
                        "?create_timestamp")
                .predicate(new FilterNullContentTypes());
sritchie commented 9 years ago

Hey @dkincaid, is this still an issue? Would love a pull req if you guys have come across a workaround. Thanks!