gabrielfalcao / sure

sophisticated automated test library and runner
https://sure.readthedocs.io
GNU General Public License v3.0
699 stars 74 forks source link

Sure hijacks ipdb interactive debugging drop-in point #141

Open aguynamedben opened 7 years ago

aguynamedben commented 7 years ago

Issue Type

Versions & Configuration

Steps to reproduce (Expected and Actual Results)

I'm using behave and splinter with sure's expect. I replaced this step:

@then('it will display the motto')
def step_impl(context):
    foo = 'bar'
    assert context.browser.is_text_present('Cool beans.')

With this implementation:

@then('it will display the motto')
def step_impl(context):
    expect(context.browser.is_text_present('Cool beans.')).to.be.true

This tests is known to fail. The problem arises when the failure occurs. As recommended in behave's docs, I have my test runner (environment.py) setup to behave to open ipdb when a step fails.

def debug_step_error(step):
    import ipdb
    ipdb.post_mortem(step.exc_traceback)

def after_step(context, step):
    if step.status == 'failed':
        screenshot_failure(context, step)

        if BEHAVE_DEBUG_ON_ERROR:
            debug_step_error(step)

The problem is that sure's internal assert hi-jacks the line number ipdb drops me into:

Expected Result (same as what happens with the traditional assert)

Saving screenshot of failure to tmp/failure_and-it-will-display-the-motto_20171024-112855.png
> /Users/ben/code/bdd/iapi/features/steps/tutorial.py(22)step_impl()
     21     foo = 'bar'
---> 22     assert context.browser.is_text_present('Cool beans.')
     23

Actual Result (sure hijacks drop-in point)

Saving screenshot of failure to tmp/failure_and-it-will-display-the-motto_20171024-114845.png
> /Users/ben/code/bdd/env-iapi/lib/python3.6/site-packages/sure/__init__.py(567)ok()
    566             msg = 'expected `{0}` to be truthy'.format(self.obj)
--> 567             assert bool(self.obj), msg
    568

I have to type u 3 times to go up, up, up to get to my own code.

Saving screenshot of failure to tmp/failure_and-it-will-display-the-motto_20171024-114845.png
> /Users/ben/code/bdd/env-iapi/lib/python3.6/site-packages/sure/__init__.py(567)ok()
    566             msg = 'expected `{0}` to be truthy'.format(self.obj)
--> 567             assert bool(self.obj), msg
    568

ipdb> u
> /Users/ben/code/bdd/env-iapi/lib/python3.6/site-packages/sure/__init__.py(384)wrapper()
    383     def wrapper(self, *args, **kw):
--> 384         value = func(self, *args, **kw)
    385         msg = "{0}({1}) failed".format(

ipdb> u
> /Users/ben/code/bdd/env-iapi/lib/python3.6/site-packages/sure/__init__.py(434)__getattr__()
    433     def __getattr__(self, name):
--> 434         return getattr(self._ab, name)
    435

ipdb> u
> /Users/ben/code/bdd/iapi/features/steps/tutorial.py(23)step_impl()
     22     foo = 'bar'
---> 23     expect(context.browser.is_text_present('Cool beans.')).to.be.true
     24

ipdb>

Is there any way to get sure and ipdb playing more nicely together?

Thanks for the time you spend working on this library. As a huge fan of RSpec/Capybara/Chai, it's godsend to have this syntax available via Python.

timofurrer commented 7 years ago

Thanks for the report! Always happy about good descriptions in issues :+1:

Is there any way to get sure and ipdb playing more nicely together?

On top of my head I have absolutely no idea. I'll investigate when I find some time. @gabrielfalcao Do you have experienced this?

Thanks for the time you spend working on this library. As a huge fan of RSpec/Capybara/Chai, it's godsend to have this syntax available via Python.

Good to hear! Thanks :beers:

gabrielfalcao commented 7 years ago

@aguynamedben great issue report indeed 👏

This is an interesting problem. I'll investigate the ipdb module to find a good solution, maybe it can be wrapped around pdb.Pdb and [explicitly declare what modules to skip](import pdb; pdb.Pdb(skip=['django.*']).set_trace()).

Something like:

import ipdb

def debug_step_error(step):
    ipdb.IPdb(skip=['sure.*']).post_mortem(step.exc_traceback)