search

nathanmarz / cascalog

Data processing on Hadoop without the hassle.
Other
1.38k stars 178 forks source link

ArityException: Wrong number of args (3) passed to: platform$eval4722$fn adding tuple to output collector #215

Closed dkincaid closed 9 years ago

dkincaid commented 10 years ago

On upgrading to Cascalog 2.0 the following exception is being thrown by a query. Line 121 of com.idexx.lambda.hadoop.jobs.assembler.EntityAssembler is:

                    fn_call.getOutputCollector().add(new Tuple(pimsHashId, trueAsOfSecs, valueType, value));
cascading.pipe.OperatorException: [d9b750b1-28f6-4c53-a32...][cascading.pipe.assembly.AggregateBy.initialize(AggregateBy.java:564)] operator Each failed executing operation
    at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:107)
    at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:39)
    at cascading.flow.stream.FilterEachStage.receive(FilterEachStage.java:73)
    at cascading.flow.stream.FilterEachStage.receive(FilterEachStage.java:34)
    at cascading.flow.stream.FunctionEachStage$1.collect(FunctionEachStage.java:80)
    at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:119)
    at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:107)
    at com.idexx.lambda.hadoop.jobs.assembler.EntityAssembler$ExtractFields.operate(EntityAssembler.java:121)
    at cascalog.CascalogFunctionExecutor.operate(CascalogFunctionExecutor.java:41)
    at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:99)
    at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:39)
    at cascading.flow.stream.FilterEachStage.receive(FilterEachStage.java:73)
    at cascading.flow.stream.FilterEachStage.receive(FilterEachStage.java:34)
    at cascading.flow.stream.FunctionEachStage$1.collect(FunctionEachStage.java:80)
    at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:119)
    at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:107)
    at cascading.operation.Identity$2.operate(Identity.java:137)
    at cascading.operation.Identity.operate(Identity.java:150)
    at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:99)
    at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:39)
    at cascading.flow.stream.SourceStage.map(SourceStage.java:102)
    at cascading.flow.stream.SourceStage.run(SourceStage.java:58)
    at cascading.flow.hadoop.FlowMapper.run(FlowMapper.java:127)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:418)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:211)
Caused by: clojure.lang.ArityException: Wrong number of args (3) passed to: platform$eval4722$fn
    at clojure.lang.AFn.throwArity(AFn.java:437)
    at clojure.lang.AFn.invoke(AFn.java:47)
    at clojure.lang.AFn.applyToHelper(AFn.java:167)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.lang.AFunction$1.doInvoke(AFunction.java:29)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at cascalog.aggregator.ClojureMonoidFunctor.applyPrepareFn(ClojureMonoidFunctor.java:35)
    at cascalog.aggregator.ClojureMonoidFunctor.prepare(ClojureMonoidFunctor.java:29)
    at cascalog.aggregator.ClojureMonoidFunctor.prepare(ClojureMonoidFunctor.java:11)
    at cascalog.aggregator.FoldFunctor.aggregate(FoldFunctor.java:34)
    at cascading.pipe.assembly.AggregateBy$CompositeFunction.operate(AggregateBy.java:344)
    at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:99)
    ... 25 more
Quantisan commented 10 years ago

could you show us the query please?

dkincaid commented 10 years ago

I've put the full code at https://gist.github.com/dkincaid/9e608cc5e0e3b1d3e283

I will need to remove it once you've had a chance to look it over. Thanks.

Quantisan commented 10 years ago

@dkincaid thanks for bringing these issues to our attention Unfortunately, I can't run the code so I can't work with it. Could you narrow the problem perhaps into a toy query please?

dkincaid commented 10 years ago

Ok. I'll see if I can make it fail with a simpler query.

dkincaid commented 9 years ago

We're trying to upgrade to Cascalog 2 again and having this same issue. Could someone at least tell me what exactly is happening at line 35 of ClojureMonoidFunctor.applyPrepareFn()?