circleci / bond

spying for tests
127 stars 28 forks source link

`with-spy` doesn't seem to work on tests with `:inline` definitions in metadata #65

Open E-A-Griffin opened 1 year ago

E-A-Griffin commented 1 year ago

As far as I can tell, calling with-spy (and with-stub!) seem to not work as expected on functions that have an inline definition (i.e. a function with :inline defined in the metadata).

(t/deftest a-test 
  (t/is (= 1 (count (bond/with-spy [inc]
                     (inc 1)
                     (bond/calls inc))))))
;; => #'user/a-test
user> (a-test)
​
FAIL in (a-test) (x.clj:226)
expected: (= 1 (count (bond/with-spy [inc] (inc 1) (bond/calls inc))))
  actual: (not (= 1 0))
;; => nil

(t/deftest a-test 
  (t/is (= 1 (count (bond/with-spy [identical?]
                     (identical? 1 2)
                     (bond/calls identical?))))))
;; => #'user/a-test
user> (a-test)
​
FAIL in (a-test) (x.clj:226)
expected: (= 1 (count (bond/with-spy [identical?] (identical? 1 2) (bond/calls identical?))))
  actual: (not (= 1 0))
;; => nil

;; NOTE: `=` is only inlined for its 2 argument form

(t/deftest a-test 
  (t/is (= 1 (count (bond/with-spy [=]
                     (= 1 2)
                     (bond/calls =))))))
;; => #'user/a-test
user> (a-test)
​
FAIL in (a-test) (x.clj:226)
expected: (= 1 (count (bond/with-spy [=] (= 1 2) (bond/calls =)))
  actual: (not (= 1 0))
;; => nil

;; Passes for non-inlined form
(t/deftest a-test 
  (t/is (= 1 (count (bond/with-spy [=]
                     (= 1 2 3)
                     (bond/calls =))))))
;; => #'user/a-test
user> (a-test)
;; => nil
gordonsyme commented 1 year ago

with-spy and with-stub use with-redefs in their implementation to temporarily swap out the root binding of the stubbed var with one that tracks calls, so that when that var is accessed the replacement function is read rather than the original function.

Inline metadata gives the compiler the option of replacing a call to the function with the expansion of the :inline metadata instead.

Since the function call has been replaced with the inline body there is no var access so it doesn't matter what the root binding of the var currently is, the inlined code is what gets executed.

E.g. with this code

(ns inline.core)

;; definline is a macro convenience to generate a function
;; var with the same body and :inline metadata
(definline foo
  []
  `(println "foo"))

(defn bar
  []
  (println "bar")
  (foo))

The body of bar may get expanded by the compiler to the equivalent of:

(defn bar
  []
  (println "bar")
  (println "foo"))

And so it doesn't matter how foo is stubbed or spied, it's not called.

E-A-Griffin commented 1 year ago

That makes sense, I think I know a solution to this issue though if bond would like to add support for functions with inlined arities defined based on the example here: https://clojuredocs.org/clojure.core/alter-var-root#example-5c266712e4b0ca44402ef601

I haven't developed a full proof of concept yet but I believe that a with-no-inline macro could be written and used such that a var's inline definition is dissoced from its metadata at the beginning of the macro call and then re-assoced back at the end of the macro call.