Getting error while setting Spark configuration

[mayankpahwa]

Hi,

I am trying to run a Spark application by setting the spark.driver.host in the configuration. I have written something like:

(require '[flambo.conf :as conf])

(def c (-> (conf/spark-conf)
           (conf/master "local[*]")
           (conf/app-name "flame_princess")
           (conf/set "spark.driver.host" "some-ip")))

The above code runs fine in repl but when I create a jar and try to run it, I get the following error

#error {
 :cause "No matching method found: set for class org.apache.spark.SparkConf"
 :via
 [{:type java.lang.RuntimeException
   :message "could not start [#'mecbot.ingestion.state/c] due to"
   :at [mount.core$up$fn__10906 invoke "core.cljc" 92]}
  {:type java.lang.IllegalArgumentException
   :message "No matching method found: set for class org.apache.spark.SparkConf"
   :at [clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 80]}]
 :trace
 [[clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 80]
  [clojure.lang.Reflector invokeInstanceMethod "Reflector.java" 28]
  [flambo.conf$set invokeStatic "conf.clj" 34]
  [flambo.conf$set invoke "conf.clj" 32]
  [mecbot.ingestion.state$fn__11108 invokeStatic "state.clj" 54]
  [mecbot.ingestion.state$fn__11108 invoke "state.clj" 45]
  [mount.core$record_BANG_ invokeStatic "core.cljc" 86]
  [mount.core$record_BANG_ invoke "core.cljc" 85]
  [mount.core$up$fn__10906 invoke "core.cljc" 93]
  [mount.core$up invokeStatic "core.cljc" 92]
  [mount.core$up invoke "core.cljc" 90]
  [mount.core$bring invokeStatic "core.cljc" 210]
  [mount.core$bring invoke "core.cljc" 202]
  [mount.core$start invokeStatic "core.cljc" 252]
  [mount.core$start doInvoke "core.cljc" 244]
  [clojure.lang.RestFn invoke "RestFn.java" 408]
  [mecbot.ingestion.core$_main invokeStatic "core.clj" 335]
  [mecbot.ingestion.core$_main doInvoke "core.clj" 318]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.RestFn applyTo "RestFn.java" 132]

I have also tried including Spark's dependency in the :provided profile but to no avail. Could you please point out what could be wrong. I am using Spark v2.3.1 and Flambo v0.8.2.

Thanks, Mayank