nestorsalceda / mamba

The definitive testing tool for Python. Born under the banner of Behavior Driven Development (BDD).
http://nestorsalceda.github.io/mamba
MIT License
518 stars 65 forks source link

Ensure after.each hooks run when an exception is raised? #71

Open angelsanzn opened 8 years ago

angelsanzn commented 8 years ago

When an exception is raised in an example, all after.each hooks for that example are skipped:

with description('ensuring "after" hooks get run'):
    with it('when an exception is raised in the body of the test'):
        raise Exception

    with after.each:
        print('this should be printed, but is not')
with description('ensuring "after" hooks get run'):
    with context('nested context'):
        with it('when an exception is raised in the body of the test'):
            raise Exception

        with after.each:
            print('this should be printed, but is not')

    with after.each:
        print('this should also be printed, but is not')

Is this intentional? Or should the after.each hooks be run?

srbry commented 7 years ago

This has been open a long time without comment? seems like the approach is wrong when comparing with any other testing frameworks. Would a pull request be entertained if it looked at changing this?

jes5e commented 6 years ago

I opened another issue for this accidentally because I didn't see this one. I actually submitted a pull request for it. This issue is currently making it pretty difficult to use mamba if you need to tear stuff down after every test.

nestorsalceda commented 6 years ago

I'm going to do some research about this stuff and I hope to get it done today.

Thanks for your patience.

nestorsalceda commented 6 years ago

I have seen an interesting behaviour in rspec:

  it 'raises an error' do
    raise 'Error 1'
  end

  after(:each) do
    raise 'Error 2'
  end
end

And it gives following output:


Failures:

  1) After each error raises an error
     Got 0 failures and 2 other errors:

     1.1) Failure/Error: raise 'Error 1'

          RuntimeError:
            Error 1
          # ./foo_spec.rb:3:in `block (2 levels) in <top (required)>'

     1.2) Failure/Error: raise 'Error 2'

          RuntimeError:
            Error 2
          # ./foo_spec.rb:7:in `block (2 levels) in <top (required)>'

Finished in 0.0019 seconds (files took 0.06321 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./foo_spec.rb:2 # After each error raises an error

It is raising an error composed with 2 additional errors, one for spec block and other for after.each block.

I think this is the desired behaviour for mamba and I would like to implement it.

What do you think?

Thanks.