I found a funky issue in Cascalog 2.0. When trying to use :trap with c/ops I get following exception:
java.lang.Exception: java.lang.RuntimeException: Unable to resolve symbol: compare-fn in this context, compiling:(NO_SOURCE_PATH:191:20)
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.RuntimeException: Unable to resolve symbol: compare-fn in this context, compiling:(NO_SOURCE_PATH:191:20)
at clojure.lang.Compiler.analyze(Compiler.java:6464)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3719)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6646)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3719)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6646)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$IfExpr$Parser.parse(Compiler.java:2708)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$VectorExpr.parse(Compiler.java:3126)
at clojure.lang.Compiler.analyze(Compiler.java:6447)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6100)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6632)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5217)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3846)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6642)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6632)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3719)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6646)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6632)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6100)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5217)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3846)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6642)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.eval(Compiler.java:6700)
at clojure.lang.Compiler.eval(Compiler.java:6666)
at clojure.core$eval.invoke(core.clj:2927)
at cascalog.logic.fn$fn__477$fn__485.invoke(fn.clj:169)
at cascalog.logic.fn$fn__477.invoke(fn.clj:168)
at clojure.lang.MultiFn.invoke(MultiFn.java:231)
at cascalog.logic.fn$deserialize.invoke(fn.clj:135)
at cascalog.Util.deserializeFn(Util.java:86)
at cascalog.ClojureParallelAgg.prepare(ClojureParallelAgg.java:39)
at cascalog.ClojureCombinerBase.prepare(ClojureCombinerBase.java:83)
at cascalog.ClojureBufferCombiner.prepare(ClojureBufferCombiner.java:50)
at cascading.flow.stream.OperatorStage.prepare(OperatorStage.java:284)
at cascading.flow.stream.StreamGraph.prepare(StreamGraph.java:167)
at cascading.flow.hadoop.FlowMapper.run(FlowMapper.java:113)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.RuntimeException: Unable to resolve symbol: compare-fn in this context
at clojure.lang.Util.runtimeException(Util.java:221)
at clojure.lang.Compiler.resolveIn(Compiler.java:6940)
at clojure.lang.Compiler.resolve(Compiler.java:6884)
at clojure.lang.Compiler.analyzeSymbol(Compiler.java:6845)
at clojure.lang.Compiler.analyze(Compiler.java:6427)
... 69 more
I found a funky issue in Cascalog 2.0. When trying to use
:trap
withc/ops
I get following exception:Following is the code to reproduce this issue:
I'll try to understand how to fix it. Right now it seems like it's a serialization issue.