Show fact doc strings in failure messages

[AlexBaranosky]

  (fact "do something cool"
    (+ 5 5) => 555) 

I would like to see:

FAIL: do something cool (core.clj:9) ...other report output

[AlexBaranosky]

For this to work the expect macro needs to take an optional doc-string.

Currently we identify things that should be wrapped in an expect form by whether they look like:

(+ 1 2) => 3

Thus,

"basic arithmetic works"
(+ 1 2) => 3

resolves to an expect like:

(expect (+ 1 2) => 3) ;; ignoring the string before it.

Looks like we need to change the code that identifies arrow-forms, to also identify an arrow form anything that looks like:

(+ 1 2) => 3

OR

"words..."
(+ 1 2) => 3

And then in the second case add a doc-string to the expect we wrap it in:

(expect "words..." (+ 1 2) => 3)

I've got everything from the expect level down working, but the real trick is changing the way these arrow forms get translated to expects.

[AlexBaranosky]

Ahhh crud.

This might be more sticky than you'd think.

Take this example, how will the code that wraps these forms in expect forms know that "evalue" is not in fact a doc-string to (expect "evalue" (f 2) => (+ 2 2)) ?

             '( (f 1) => [1] :ekey "evalue"
                (f 2) => (+ 2 2)
                (provided (g 3) => 3
                          (g 4) => 4 :pkey "pvalue")
                (f 5) => truthy)

[marick]

A conceptual difficulty I had with this idea way back when Ben Mabey first asked for it is that my facts often have sequences of checks interspersed with doc strings. The doc string is supposed to apply until the next doc string.

Serendipitously (but now documented by tests), you can nest facts:

(facts "general statement"
   (facts "sub-statement" ...)
   (facts "sub-statement" ...)

That given, I'd be tempted to handle doc strings well before the point where expect forms are created. Strings other than those immediately following fact do not count as doc strings and will never get printed.

This would be consistent with Ruby test frameworks like Shoulda or Rspec.

As far as implementation goes, it occurs to me this could be somehow merged with the dynamic scoping that's used for background prerequisites. At the point a failure is reported, you look up the dynamic scope to see the nested "descriptive contexts".

The whole background infrastucture/code needs to be rewritten, I think. Not only is there an outstanding double-evaluation bug, the code is kind of confusing, even to me, partly because there are scopes that produce variable bindings that affect prerequisites in other scopes

I've been thinking, in a vague way, of some sort of backward chaining approach. [I tried to explain it, but I realized it's too vague even for that.]

[AlexBaranosky]

I will concur that the background stuff is the hardest to follow.

I think the nested doc string idea is a good one. People are pretty used to that style, and tend to like it.

Maybe we implement it as a simple stack of messages, that we push onto when we start evaluating a fact, and pop off of when we end a fact? And when we generate the report data, just concat (in a prettified way) all the messages on the current stack.

When I say it like that it doesn't sound so tough...... hmmm......

[AlexBaranosky]

Using a global dynamic var stack would also work for the background info, which I think might have been what you were getting at all along.

[AlexBaranosky]

If it sounds architecturally kosher, I'd like to take this issue and see if I can finish what I started a few days ago, but instead using a dynamic var to hold the stack of context messages.

I'd keep it separate from the idea of backgrounds, as that would require refactoring. If this goes well, then a clearer idea might develop of how backgrounds could be modeled in a similar way.

[marick]

That sounds fine to me.

On Dec 20, 2011, at 7:12 PM, AlexBaranosky wrote:

If it sounds architecturally kosher, I'd like to take this issue and see if I can finish what I started a few days ago, but instead using a dynamic var to hold the stack of context messages.

I'd keep it separate from the idea of backgrounds, as that would require refactoring. If this goes well, then there might be a clearer idea of how backgrounds could be modeled in a similar way.

---

Reply to this email directly or view it on GitHub: https://github.com/marick/Midje/issues/74#issuecomment-3228115

---

Brian Marick, Artisanal Labrador Now working at http://path11.com Contract programming in Ruby and Clojure Occasional consulting on Agile

[AlexBaranosky]

I got a version of this working on a local branch. Its not well tested end-to-end, as I was spiking most of it. As soon as I'm happy with how its tested it should be good to go.

[AlexBaranosky]

Just pushed an implementation of this that has syntax like this:

(facts "about mathematics"
  (fact "addition is commutative"
    (+ 1 2) => 3
    (+ 2 1) => 4)  ;; WRONG

  (fact "multiplication is commutative"
    (* 2 5) => 10
    (* 5 2) => 11)) ;; WRONG

;; These would report as:

;; FAIL: "about mathematics, addition is commutative" at (core.clj:995)
;; ...
;; FAIL: "about mathematics, multiplication is commutative" at (core.clj:999)
;; ...