search

CleanCut / green

Green is a clean, colorful, fast python test runner.
MIT License
785 stars 75 forks source link

NameError not catched by "green" but by "unittest" #223

Closed buhtz closed 4 years ago

buhtz commented 4 years ago

I try to understand the differences in the output between green and unittest because the latter shows me a very important exception. Please look for the line NameError: name 'bascis' is not defined in that code:


python3 -m unittest
>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.LINTGHQGA6M"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.LINTGHQGA6M"
Feeds file does not exist. Create a default file...
.E>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.T38ZX3J"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.T38ZX3J"
E>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.C4UJYQK8NZIHPC4S3AIIX3DK"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.C4UJYQK8NZIHPC4S3AIIX3DK"
E
======================================================================
ERROR: tearDownClass (tests.test_data.TestFeedsData)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/share/work/Feedybus/tests/test_data.py", line 15, in tearDownClass
    helper.clear_profile()
  File "/home/user/share/work/Feedybus/tests/helper.py", line 41, in clear_profile
    bascis.log_filename()]:
NameError: name 'bascis' is not defined

======================================================================
ERROR: setUpClass (tests.test_feed.TestFeed)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/share/work/Feedybus/tests/test_feed.py", line 32, in setUpClass
    data.FeedsData()
  File "/home/user/share/work/Feedybus/feedybus/data.py", line 65, in __init__
    raise Exception('Only one instance of FeedsData can exist.')
Exception: Only one instance of FeedsData can exist.

======================================================================
ERROR: setUpClass (tests.test_group.TestGroup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/share/work/Feedybus/tests/test_group.py", line 37, in setUpClass
    data.FeedsData()
  File "/home/user/share/work/Feedybus/feedybus/data.py", line 65, in __init__
    raise Exception('Only one instance of FeedsData can exist.')
Exception: Only one instance of FeedsData can exist.

----------------------------------------------------------------------
Ran 1 test in 0.009s

FAILED (errors=3)

Now you see the output of green where this error (a simple typo by the way) is missing but very important because it is in the tearDownClass() methode.

green --processes 1 -vvv -a
Green 3.0.0, Coverage 4.5.4, Python 3.7.3

>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.IKRDB50RJT"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.IKRDB50RJT"
Feedybus.tests.test_data
  TestFeedsData
.   test_singleton
>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.W65ZBYT"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.W65ZBYT"
green.suite
  GreenTestSuite
E   setUpClasste.GreenTestSuite
>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.RVKMEUG06Y56G"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.RVKMEUG06Y56G"
E   setUpClasste.GreenTestSuite

Error in unittest.suite._ErrorHolder.setUpClass
  File "/usr/local/lib/python3.7/dist-packages/green/suite.py", line 105, in _handleClassSetUp
    setUpClass()
  File "/home/user/share/work/Feedybus/tests/test_feed.py", line 32, in setUpClass
    data.FeedsData()
  File "/home/user/share/work/Feedybus/feedybus/data.py", line 65, in __init__
    raise Exception('Only one instance of FeedsData can exist.')
Exception: Only one instance of FeedsData can exist.

Error in unittest.suite._ErrorHolder.setUpClass
  File "/usr/local/lib/python3.7/dist-packages/green/suite.py", line 105, in _handleClassSetUp
    setUpClass()
  File "/home/user/share/work/Feedybus/tests/test_group.py", line 37, in setUpClass
    data.FeedsData()
  File "/home/user/share/work/Feedybus/feedybus/data.py", line 65, in __init__
    raise Exception('Only one instance of FeedsData can exist.')
Exception: Only one instance of FeedsData can exist.

Ran 3 tests in 0.209s using 1 process

FAILED (errors=2, passes=1)

Maybe I missunderstood or missconfigured something here?

CleanCut commented 4 years ago

Does the error occur with the latest version of Green? (3.1.3 at the moment)

Is your code publicly available? (Can I look at it?)

buhtz commented 4 years ago

The code is not public - but just because of the current quality state. So no problem for me if someone else looks into it. I attached an archive and I am open for all suggestions about improving that code.

Feedybus_for_green.zip

Currently I am in a big refactoring phase. Creating tests for that code is also part of that phase. It is the first time I try to use tests in a "real" coding project. btw: The official (but empty) repository of the projects can be found and a very cool and free code hosting plattform https://codeberg.org/buhtz/Feedybus .

Since my first posting I modified the code. So please see here the output I can reproduce with the code from the attached archive.

This is unittest run

$ python3 -m unittest discover -v
>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.60M0A7Y9T1324TWWIKHJYML"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.60M0A7Y9T1324TWWIKHJYML"
Feeds file does not exist. Create a default file...
test_singleton (tests.test_data.TestFeedsData) ... ok
<<< Removed directory tree /home/user/.local/share/feedybus/favicons.UNITTEST.60M0A7Y9T1324TWWIKHJYML.
<<< Removed directory tree /home/user/.local/share/feedybus/entries.UNITTEST.60M0A7Y9T1324TWWIKHJYML.
ERROR
>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.GBK2A4IUNE9SMPJARP1MZAFH"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.GBK2A4IUNE9SMPJARP1MZAFH"
ERROR

======================================================================
ERROR: tearDownModule (tests.test_data)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/Downloads/Feedybus_for_green/tests/test_data.py", line 26, in tearDownModule
    helper.clear_profile()
  File "/home/user/Downloads/Feedybus_for_green/tests/helper.py", line 46, in clear_profile
    bascis.log_filename()]:
NameError: name 'bascis' is not defined

======================================================================
ERROR: setUpModule (tests.test_group)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/Downloads/Feedybus_for_green/tests/test_group.py", line 31, in setUpModule
    data.FeedsData()
  File "/home/user/Downloads/Feedybus_for_green/feedybus/data.py", line 65, in __init__
    raise Exception('Only one instance of FeedsData can exist.')
Exception: Only one instance of FeedsData can exist.

----------------------------------------------------------------------
Ran 1 test in 0.008s

FAILED (errors=2)

This is green:


green --process 1 -vvv -a
Green 3.0.0, Coverage 4.5.4, Python 3.7.3

>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.H1K"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.H1K"
Feedybus_for_green.tests.test_data
  TestFeedsData
.   test_singleton
>>> Create "/home/user/.local/share/feedybus/entries.UNITTEST.G3T3G5E0BDHW0L"
>>> Create "/home/user/.local/share/feedybus/favicons.UNITTEST.G3T3G5E0BDHW0L"
green.suite
  GreenTestSuite
E   setUpModulee.GreenTestSuite

Error in unittest.suite._ErrorHolder.setUpModule
  File "/usr/lib/python3.7/unittest/suite.py", line 200, in _handleModuleFixture
    setUpModule()
  File "/home/user/Downloads/Feedybus_for_green/tests/test_group.py", line 31, in setUpModule
    data.FeedsData()
  File "/home/user/Downloads/Feedybus_for_green/feedybus/data.py", line 65, in __init__
    raise Exception('Only one instance of FeedsData can exist.')
Exception: Only one instance of FeedsData can exist.

Ran 2 tests in 0.107s using 1 process

FAILED (errors=1, passes=1)

The "problem" with the code is in the file tests/helper.py at line 46 - where a simple misspelling occured.

CleanCut commented 4 years ago

This one was tricky! Thanks to your bug report, I was able to identify and fix a bug that has been present in Green for years that has masked errors in module- or class-level teardowns! ❤️

The fix is included in Green 3.1.4 (just released).

CleanCut commented 4 years ago

Oh, and it's the 🥧 release, how fun! 😁

buhtz commented 4 years ago

I can confirm that this is fiex in 3.1.4. Thanks a lot.

btw: I would like to see green as part of the official debian repository. I know it is not easy. ;)

CleanCut commented 4 years ago

btw: I would like to see green as part of the official debian repository. I know it is not easy. ;)

I'm glad you like it that much!