beanbaginc / kgb

Python function spy support for unit tests
MIT License
48 stars 5 forks source link

ExistingSpyError: The function <function foobar at ......> has already been spied on. #5

Closed zorchan closed 4 years ago

zorchan commented 4 years ago

I am using DDT for my unit tests. The basic idea is that my single piece of test logic will be injected with multiple test parameters and being executed multiple times.

Recently, I started to look at Python spy libraries and I am trying your SpyAgency. The DDT library above repeats my test case as long as I have 2 or 2+ tuples of parameters. The first iteration of test ran OK but starting from the 2nd one, the spy_on() threw this:

ExistingSpyError: The function <function foobar at ......> has already been spied on. Here is where that spy was set up:

... (omitted)

You may have encountered a crash in that test preventing the spy from being unregistered. Try running that test manually.

In such situation, what will be your recommendation to handle this case? Thank you.

(Btw, thanks for filling this feature gap in Python)

chipx86 commented 4 years ago

Hi! So I haven't used DDT myself, so KGB is definitely not built to be specifically compatible with it, but glancing at the DDT code, I feel like it should work.

DDT just seems to look for test functions that contain @data decorators and create a copy of the test function for each data value. This means that in the end, it's equivalent to a bunch of test functions. (I may be very wrong, but it looks like a simple case of @ddt on the class and @data on the test functions behaves this way.)

Therefore, you should get a call to setUp() before each test and tearDown() after.

If this is the case, KGB should work as expected. You'd want to set up your test suite's class like so:

@ddt
class MyTests(SpyAgency, TestCase):
    ...

SpyAgency is built to work as a mixin, automatically unspying all spies whenever a test completes.

You can, of course, manually create a SpyAgency instance and spy using that, but you're then responsible for cleaning that up afterwards. We recommend the mixin approach.

Is that the approach you're using now? I'm guessing at the causes, since I don't have your code, but if you're not using it as a mixin, I suspect that's the main cause. That or you've overridden setUp() and tearDown() but haven't called the parent methods for each.

zorchan commented 4 years ago

Thanks for the quick reply above. I am actually creating my new decorator to call your SpyAgency so that I have the code pattern highly similar to the existing @patch. i.e.

@data( ... )
@myspy( ... )
@patch('...', return_value=...)
def test_something(self, ...)

Within my code, I have to do a try-except inside @myspy order to avoid the exception in my original message.

    def wrapper(*args, **kwargs):
        sa = SpyAgency()
        try:
            sa.spy_on(spyingTarget)
        except:
            spyingTarget.reset_calls()
chipx86 commented 4 years ago

You're really going to want to call .unspy() at some point. If you let the class mix SpyAgency in, it seems like it'd do the work for you?

My reading of the DTT code is that any test_* method that uses the @data decorator just becomes multiple test_* methods that would each begin with a setUp() and end with a tearDown(), which means that the SpyAgency mixin should be able to manage all spies the way it was intended.

zorchan commented 4 years ago

Ok, ic. That helps. Thanks again.