slagyr / speclj

pronounced "speckle": a TDD/BDD framework for Clojure.
MIT License
459 stars 58 forks source link

Code in around not executing on spec failures #117

Closed dcmoore closed 9 years ago

dcmoore commented 9 years ago

Any code after the characteristic doesn't execute if the characteristic fails. This can make test cleanup hard, and sometimes cause test pollution.

(around [it]
  (let [original-value @stateful-var]
    (it)
    (reset! stateful-var original-value)))
trptcolin commented 9 years ago

I didn't expect this behavior - I tend to use try/finally for this reason. But I double-checked this in RSpec, and sure enough, it does things the way you want it to, running specs even if examples fail.

Using try/finally in your around is a workaround in the short term, but perhaps this should change.

I dug into this a bit this morning, and I have a potential solution, but it's a bit janky and has some assumptions:

  1. If there are nested arounds, it's OK to skip rescuing exceptions that come *directly from the around code), such that a nested around failure could prevent cleanup in outer ones.
  2. People aren't already depending on the current behavior, where you can rescue exceptions propagating from the it. I've definitely used try/finally, which would continue to work, but users of try/catch in an around would be broken by this change. Not sure if any uses like that exist.

I'd be interested to hear others' thoughts on this. Maybe there's a bigger design change that would make this simpler?

slagyr commented 9 years ago

After much discussion with @trptcolin and experimentation, we concluded that the best option is to do nothing. Will not Fix.

If you have cleanup to do, add a try/finally block to your around code:

(around [it]
  (let [original-value @stateful-var]
    (try
      (it)
      (finally
         (reset! stateful-var original-value)))))

A doc comment to this effect has been added to the source.