search

damballa / parkour

Hadoop MapReduce in idiomatic Clojure.
Apache License 2.0
257 stars 19 forks source link

ArityException Wrong number of args (3) passed to: PersistentArrayMap clojure.lang.AFn.throwArity #6

Closed arkadiuszbicz closed 9 years ago

arkadiuszbicz commented 9 years ago

Running from lein repl (require 'parkour.repl) (->> (text/dseq "test.txt") (parkour.repl/launch! (get-conf) {"mapred.reduce.tasks" 313} word-count)) I've got bellow error Warning: profile :jobjar not found.) (Warning: profile :jobjar not found.) Warning: specified :main without including it in :aot. Implicit AOT of :main will be removed in Leiningen 3.0.0. If you only need AOT for your uberjar, consider adding :aot :all into your :uberjar profile instead. Compiling parkour.example.wordcount 14/10/08 16:32:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method. Created /opt/nelly/clojure/parkourtest/target/parkourtest-0.1.jar 14/10/08 16:32:37 WARN hdfs.DomainSocketFactory: The short-circuit local reads feature is disabled because libhadoop cannot be loaded.

ArityException Wrong number of args (3) passed to: PersistentArrayMap clojure.lang.AFn.throwArity (AFn.java:429)

Project.clj (defproject parkourtest "0.1" :description "Map reduce test" :dependencies [[org.codehaus.jsr166-mirror/jsr166y "1.7.0"] [com.damballa/parkour "0.6.0"] [org.clojure/clojure "1.6.0"]]

        :profiles {:dev {:dependencies [[alembic "0.3.2"]]}
                    :provided
                   {:dependencies
                     [[org.apache.hadoop/hadoop-client "2.0.0-mr1-cdh4.2.1"]
                     [org.apache.hadoop/hadoop-core "2.0.0-mr1-cdh4.2.1"]
                      [org.apache.hadoop/hadoop-common "2.0.0-cdh4.2.1"]]}}
        :repositories [["clouderapublic" "http://repository.cloudera.com/artifactory/public/"]]

        :main parkour.example.wordcount)

wordcount.clj (defn get-conf [](doto %28conf/ig%29 %28.addResource %28new Path)) (.addResource (new Path "hdfs-site.xml")) (.addResource (new Path "mapred-site.xml")) (.set "hadoop.job.ugi" "myuser,mygroup")))

llasram commented 9 years ago

You're passing launch! too many arguments. The second argument should be the entry-point function. It being a map instead leads to the argument you're seeing, when the map is invoked a function.