replumb/read-eval-call doesn't support `:refer`

[viebel]

When calling replumb/read-eval-call with :refer inside the source, it doesn't work.

For instance,

(defn no-op [file-url src-cb]
  (src-cb ""))

(def repl-opts-noop (merge (replumb/options :browser
                                             ["/dbg/js" "/js/compiled/out"]
                                             io/no-op)
                            {:warning-as-error false
                             :context :statement
                             :verbose false}))
(replumb/read-eval-call repl-opts-noop identity "(ns my.aa (:require [clojure.set :refer [union]]) )")

; => {:success? false, :form (ns my.aa (:require [clojure.set :refer [union]])), :warning nil, :error #error {:message "Could not parse ns form my.aa", :data {:tag :cljs/analysis-error}, :cause #error {:message "Invalid :refer, var var clojure.set/union does not exist", :data {:tag :cljs/analysis-error}}}}

While it works fine with :as

(replumb/read-eval-call repl-opts-noop identity "(ns my.aa (:require [clojure.set :as set])) (set/union)")
; => {:success? true, :form (ns my.aa (:require [clojure.set :as set])), :warning nil, :value "#{}"}

[arichiardi]

Uhm, it should work, do you have the clojure source of clojure.set in /dbg/js ?

[arichiardi]

If yes, can you add :verbose true to the options and post here the call log?

[viebel]

My load function is a dummy function that returns an empty string. It works fine when requiring clojure.set with :as (because clojure.set is already loaded)

Here is the log:

(replumb/read-eval-call (merge repl-opts-noop {:verbose true}) identity "(ns my.aa (:require [clojure.set :refer [union]]))
 (set/union)")
Calling eval-str on (ns my.aa (:require [clojure.set :refer [union]])) with options {:read-file-fn! <hidden function>, :init-fns <hidden func
tion>, :write-file-fn! <hidden function>, :verbose true, :warning-as-error false, :context :statement, :load-fn! <hidden function>, :target :
default, :src-paths [/dbg/js /js/compiled/out]}
Evaluation returned:  {:error #error {:message Could not parse ns form my.aa, :data {:tag :cljs/analysis-error}, :cause #error {:message Inva
lid :refer, var var clojure.set/union does not exist, :data {:tag :cljs/analysis-error}}}}
Calling back!
 {:opts
 {:read-file-fn! "<hidden function>",
  :init-fns "<hidden function>",
  :write-file-fn! "<hidden function>",
  :verbose true,
  :warning-as-error false,
  :context :statement,
  :load-fn! "<hidden function>",
  :target :default,
  :src-paths ["/dbg/js" "/js/compiled/out"]},
 :data
 {:form (ns my.aa (:require [clojure.set :refer [union]])),
  :ns my.aa,
  :target :default,
  :on-success-fn! "<hidden function>"},
 :res
 {:error
  #error {:message "Could not parse ns form my.aa", :data {:tag :cljs/analysis-error}, :cause #error {:message "Invalid :refer, var var cloju
re.set/union does not exist", :data {:tag :cljs/analysis-error}}}}}

{:success? false, :form (ns my.aa (:require [clojure.set :refer [union]])), :warning nil, :error #error {:message "Could not parse ns form my
.aa", :data {:tag :cljs/analysis-error}, :cause #error {:message "Invalid :refer, var var clojure.set/union does not exist", :data {:tag :clj
s/analysis-error}}}}

[arichiardi]

Yes you are right, it is tricky though because they are loaded but they might not have metadata (analysis cache). So if I add that straight to skip-load?, the require and consequent load of the actual files will never be triggered...

[arichiardi]

Maybe we should try to load those namespaces at the beginning during the init phase and consider them as "loaded" if not present in any :src-paths. If somebody wants to be able to add for instance clojure.set files at runtime, she will need to copy them and force an init, but I see this last use case not very useful in general.

[viebel]

How do I add that straight to skip-load??

What modifications should I make to my code? Please give me an explicit piece of code :)

Maybe using repl/eval-str* ?

[arichiardi]

Skip load is part of load-fn so the problem might actually be in cljs.js. I am investigating.

[viebel]

@arichiardi did you discover something interesting?

[arichiardi]

Kind of delayed, probably I will today :)

[arichiardi]

So probably, maybe, attentively it boils down to this and then this.

I tried the following test and it works:

(h/read-eval-call-test (assoc e/*target-opts*
                              :verbose true
                              :load-fn! (fn [m cb]
                                          (cb {:lang :clj
                                               :source "(ns clojure.set) (defn union [] nil)"})))
  ["(ns my.aa (:require [clojure.set :refer [union]]))"]
  (let [out (unwrap-result @_res_)]
    (is (success? @_res_) (str _msg_ "should succeed"))
    (is (valid-eval-result? out) (str _msg_ "should be a valid result"))))

[arichiardi]

I have also noticed that planck handles the case separately and it actually always loads the cache that is probably what is missing here:

https://github.com/mfikes/planck/blob/master/planck-cljs/src/planck/repl.cljs#L626

[viebel]

I cannot call read-eval-call-test from my project as test-helpers namespace is not exported.

Is there a way to use the piece of code you share inside my project?

[arichiardi]

Uhm, well that's just a macro around read-eval-call really, it was just to show you that you need the clojure.set source on the classpath now, waiting for a fix....

[arichiardi]

So @viebel the solution is:

• drop load-fn! because if you use it you are not going to be able to take advantage of replumb nice features :smile:
• Use the new replumb.core/options with an actual io function for reading
• Pass :cache {:src-paths-lookup? true} along the option map so that you can load the analysis cache

[viebel]

I'm not yet able to make it work. I have tried the following

(defn no-op [file-url src-cb]
  (src-cb ""))
(def repl-opts-noop (merge (replumb/options :browser
                                             ["/dbg/js" "/js/compiled/out"]
                                             no-op)
                            {:warning-as-error false
                             :context :statement
                             :cache {:src-paths-lookup? true}
                             :verbose false}))
(replumb/read-eval-call (merge repl-opts-noop {:verbose true}) identity "(ns my.aa (:require [clojure.set :as set :refer [union]])) (union)")

And I get this log

Calling eval-str on (ns my.aa (:require [clojure.set :as set :refer [union]])) with options {:read-file-fn! <hidden function>, :init-fns <hid
den function>, :write-file-fn! <hidden function>, :verbose true, :cache {:src-paths-lookup? true}, :warning-as-error false, :context :stateme
nt, :load-fn! <hidden function>, :target :default, :src-paths [/dbg/js /js/compiled/out]}
Evaluation returned:  {:error #error {:message Could not parse ns form my.aa, :data {:tag :cljs/analysis-error}, :cause #error {:message Inva
lid :refer, var var clojure.set/union does not exist, :data {:tag :cljs/analysis-error}}}}
Calling eval-str on (ns my.aa (:require [clojure.set :as set :refer [union]])) with options {:read-file-fn! <hidden function>, :init-fns <hid
den function>, :write-file-fn! <hidden function>, :verbose true, :warning-as-error false, :context :statement, :load-fn! <hidden function>, :
target :default, :src-paths [/dbg/js /js/compiled/out]}
Evaluation returned:  {:error #error {:message Could not parse ns form my.aa, :data {:tag :cljs/analysis-error}, :cause #error {:message Inva
lid :refer, var var clojure.set/union does not exist, :data {:tag :cljs/analysis-error}}}}
Calling back!
 {:opts
 {:read-file-fn! "<hidden function>",
  :init-fns "<hidden function>",
  :write-file-fn! "<hidden function>",
  :verbose true,
  :cache {:src-paths-lookup? true},
  :warning-as-error false,
  :context :statement,
  :load-fn! "<hidden function>",
  :target :default,
  :src-paths ["/dbg/js" "/js/compiled/out"]},
 :data
 {:form (ns my.aa (:require [clojure.set :as set :refer [union]])),
  :ns cljs.user,
  :target :default,
  :on-success-fn! "<hidden function>"},
 :res
 {:error
  #error {:message "Could not parse ns form my.aa", :data {:tag :cljs/analysis-error}, :cause #error {:message "Invalid :refer, var var cloju
re.set/union does not exist", :data {:tag :cljs/analysis-error}}}}}

:success? false, :form (ns my.aa (:require [clojure.set :as set :refer [union]])), :warning nil, :error #error {:message "Could not parse ns
 form my.aa", :data {:tag :cljs/analysis-error}, :cause #error {:message "Invalid :refer, var var clojure.set/union does not exist", :data {:
tag :cljs/analysis-error}}}}
klipse.compiler=> Calling back!
 {:opts
 {:read-file-fn! "<hidden function>",
  :init-fns "<hidden function>",
  :write-file-fn! "<hidden function>",
  :verbose true,
  :warning-as-error false,
  :context :statement,
  :load-fn! "<hidden function>",
  :target :default,
  :src-paths ["/dbg/js" "/js/compiled/out"]},
 :data
 {:form (ns my.aa (:require [clojure.set :as set :refer [union]])),
  :ns foo.core$macros,
  :target :default,
  :on-success-fn! "<hidden function>"},
 :res
 {:error
  #error {:message "Could not parse ns form my.aa", :data {:tag :cljs/analysis-error}, :cause #error {:message "Invalid :refer, var var clojure.set/union does not exist", :data {:tag :cljs/analysis-error}}}}}

[arichiardi]

You have to specify a io/read-file! in replumb.core/options or you won't be able to load the analysis cache. In some way ClojureScript needs to have it for :refer to work and there are two ways to achieve this:

1. To return the clojure.set namespace in :source of your load-fn!
2. To read and load the analysis cache from somewhere (in our case from your src

[viebel]

Great. It works with option #2.

I don't understand how to implement #1.

Also, is there a way to use macros inside a repl? It sounds tricky because macros are defined inside clj files.

[arichiardi]

Great! Probably this explains it better, I will close this one if you don't mind. http://blog.fikesfarm.com/posts/2015-09-07-messing-with-macros-at-the-repl.html