HCADatalab / powderkeg

Live-coding the cluster!
Eclipse Public License 1.0
159 stars 23 forks source link

Unable to resolve spec definition #29

Open mtijerina opened 7 years ago

mtijerina commented 7 years ago

When I define a spec with s/def, write a function that uses this spec somehow, and then use this function within keg/rdd, I am getting a "Unable to resolve spec" error. As a very simplified example:

(s/def ::odd-int (s/and int? odd?))

(defn nums-are-odd-ints? [num]
  (s/valid? ::odd-int num))

(into [] (keg/rdd [1 3 5 7 9 11] (map nums-are-odd-ints?)))
An exception was caused by: java.lang.Exception (Unable to resolve spec: :user/odd-int)

I am running powderkeg.repl in yarn-client mode.

cgrand commented 7 years ago

It is a known issue. Spec uses an atom as the repo and atoms are not synchroyacross the cluster as it makes little sense in general.

A workaround is to dirty the repo var manually.

Le lun. 27 mars 2017 à 20:50, matthew-tijerina notifications@github.com a écrit :

When I define a spec with s/def, write a function that uses this spec somehow, and then use this function within keg/rdd, I am getting a "Unable to resolve spec" error. As a very simplified example:

(s/def ::odd-int (s/and int? odd?))

(defn nums-are-odd-ints? [num] (s/valid? ::odd-int num))

(into [] (keg/rdd [1 3 5 7 9 11] (map nums-are-odd-ints?)))

An exception was caused by: java.lang.Exception (Unable to resolve spec: :user/odd-int)

I am running powderkeg.repl in yarn-client mode.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/HCADatalab/powderkeg/issues/29, or mute the thread https://github.com/notifications/unsubscribe-auth/AAC3sfhUKrV6uKw9olVD0-miqhR52dviks5rqATtgaJpZM4MqrC- .

-- On Clojure http://clj-me.cgrand.net/ Clojure Programming http://clojurebook.com Training, Consulting & Contracting http://lambdanext.eu/

cgrand commented 7 years ago

Similar to #14