unittest integration breaks when a test case fails

[symwell]

Running APPMAP=true python manage.py test in the repository https://github.com/Antylon/django-polls on branch appmap-testcase produces the error

~/src/django-polls$ git branch
* appmap-testcase
  master
~/src/django-polls$ APPMAP=true python manage.py test
System check identified no issues (0 silenced).
Traceback (most recent call last):
  File "/home/test/src/django-polls/manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
    super().run_from_argv(argv)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/core/management/commands/test.py", line 53, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/test/runner.py", line 699, in run_tests
    result = self.run_suite(suite)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/django/test/runner.py", line 641, in run_suite
    return runner.run(suite)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/unittest/runner.py", line 184, in run
    test(result)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/unittest/suite.py", line 122, in run
    test(result)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/unittest/case.py", line 651, in __call__
    return self.run(*args, **kwds)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/unittest/case.py", line 591, in run
    with outcome.testPartExecutor(self, isTest=True):
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/appmap/unittest.py", line 43, in testPartExecutor
    location = get_test_location(test_case.__class__, method_name)
  File "/home/test/.pyenv/versions/3.9.14/lib/python3.9/site-packages/appmap/unittest.py", line 22, in get_test_location
    fn = getattr(cls, method_name)
AttributeError: type object '_FailedTest' has no attribute 'tests'

[symwell]

I thought it was because

test_case id is unittest.loader._FailedTest.polls.tests

and it extracts as test id only the last token tests instead of perhaps polls.tests

method_name = test_case.id().split(".")[-1]

But after changing this logic in a few ways it didn't fix it.

[apotterri]

This actually turns out to be a weirdness in the test runner (or maybe unittest itself).

The problem is that the user added the line

from appmap.unittest import TestCase

to the top of their test file, but there's no TestCase in appmap.unttest:

ajp@Alans-MacBook-Pro django-polls % head polls/tests.py 
import datetime

from appmap.unittest import TestCase 
from django.urls import reverse
from django.utils import timezone

from .models import Question

class QuestionModelTests(TestCase):
(django-polls) appmap-testcase
ajp@Alans-MacBook-Pro django-polls % python             
Python 3.10.8 (main, Oct 14 2022, 08:08:42) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from appmap.unittest import TestCase
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'TestCase' from 'appmap.unittest' (/Users/ajp/.local/share/virtualenvs/django-polls-RIoQgNjI/lib/python3.10/site-packages/appmap/unittest.py)

Instead of telling the user this, the runner continues until it blows up with the above error.

[Antylon]

If appmap.unittest doesn't have TestCase then it would be nice if you could update your documentation with symwell's example.

I was very confused and perplexed that I would have to use appmap.unittest.TestCase since Django has its own TestCase that subclasses from unittest.TestCase.

If the example from @symwell was part of the doc then the misunderstanding wouldn't occur =)

• Example I got from @symwell shows the correct way of doing it (It was my fault for not noticing that appmap.unittest wasn't used for TestCase)
• The doc that I misunderstood.

[apotterri]

Thanks very much for your input, I agree that the doc is pretty terse. We'll get it updated with an example.