search

amperity / sparkplug

Clojure API bindings for Apache Spark
Other
28 stars 1 forks source link

Fix warnings when using keywords and sets as functions #10

Closed brandonvin closed 4 years ago

brandonvin commented 4 years ago

Seen a few times that doing something like

(->> (rdd/parallelize spark-context [{:foo 1} {:foo 2}])
     (spark/map :foo)
     (spark/collect))

results in log messages like:

19/12/16 04:20:01 WARN function.SerializableFn: Error loading namespace clojure.lang.Keyword java.io.FileNotFoundException: Could not locate clojure/lang/Keyword__init.class, clojure/lang/Keyword.clj or clojure/lang/Keyword.cljc on classpath.

I think this can be solved with a small patch to check if the symbol resolves to a class name (as opposed to a namespace), and avoid marking it to be required in that case.