llasram
commented
10 years ago There's not enough information here to help you, nor is there anything here to suggest that this is a bug in Parkour versus your code. If you would like to e-mail me directly or e-mail the Parkour mailing list, I'd be glad to help you out, but please stop opening Parkour tickets for non-Parkour problems.
Running bellow application, I have received error:
14/10/09 09:52:44 INFO parkour.graph: Launching job parkour.example.wordcount/word-count[1/1] 14/10/09 09:52:44 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 14/10/09 09:52:44 INFO input.FileInputFormat: Total input paths to process : 1 14/10/09 09:53:09 WARN parkour.graph: Job parkour.example.wordcount/word-count[1/1] failed 14/10/09 09:53:09 WARN parkour.graph: Cleaning up path /tmp/myuser-1412862763301-488491490-parkour-transient/t-61 14/10/09 09:53:09 ERROR parkour.tool: Uncaught exception: clojure.lang.ExceptionInfo: Job parkour.example.wordcount/word-count[ 1/1] failed. {:jname "parkour.example.wordcount/word-count[1/1]"} clojure.lang.ExceptionInfo: Job parkour.example.wordcount/word-count[1/1] failed. {:jname "parkour.example.wordcount/word-count [1/1]"} at clojure.core$ex_info.invoke(core.clj:4403) at clojure.lang.AFn.applyToHelper(AFn.java:156) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.core$apply.invoke(core.clj:624) at parkour.graph$fn1814$fn1815.doInvoke(graph.clj:377) at clojure.lang.RestFn.invoke(RestFn.java:397) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.RestFn.applyTo(RestFn.java:132) at clojure.core$apply.invoke(core.clj:624) at parkour.graph$graph_future$fn1676.invoke(graph.clj:26) at clojure.core$binding_conveyor_fn$fn4145.invoke(core.clj:1910) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
(ns parkour.example.wordcount (:require [clojure.string :as str] [clojure.core.reducers :as r] [parkour (conf :as conf) (fs :as fs) (mapreduce :as mr) , (graph :as pg) (toolbox :as ptb) (tool :as tool)] [parkour.io (text :as text) (seqf :as seqf)]) (:import [org.apache.hadoop.io Text LongWritable] [org.apache.hadoop.fs Path] [java.io FileInputStream] [org.apache.hadoop.conf Configuration Configurable]))
(defn word-count-m [coll](->> coll %28r/mapcat #%28str/split % #"s+"%29%29 %28r/map #%28-> [% 1]%29%29))
(defn word-count [conf lines](-> %28pg/input lines%29 %28pg/map #'word-count-m%29 %28pg/partition [Text LongWritable]%29 %28pg/combine #'ptb/keyvalgroups-r #'+%29 %28pg/output %28seqf/dsink [Text LongWritable]%29%29 %28pg/fexecute conf `word-count%29))
(defn tool [conf & inpaths](->> %28apply text/dseq inpaths%29 %28word-count conf%29 %28into {}%29 %28prn%29))
(defn get-conf [](doto %28Configuration.%29 %28.addResource %28new Path)) (.addResource (new Path "hdfs-site.xml")) (.addResource (new Path"mapred-site.xml")) (.set "hadoop.job.ugi" "mygroup,supergroup")))
(defn -main [& args](let [confi %28get-conf%29] %28binding [parkour.conf/default confi] %28let [_ %28prn %28vals %28conf/ig%29%29%29 tlres %28tool/run tool ["/tmp/arektest/in/"]%29 ] %28System/exit tlres%29%29%29))