search

JetBrains / teamcity-messages

Python Unit Test Reporting to TeamCity
https://pypi.python.org/pypi/teamcity-messages
Apache License 2.0
136 stars 81 forks source link

pytest-subtests is breaking the logs #280

Open PhilipGarnero opened 4 months ago

PhilipGarnero commented 4 months ago

As of today with Pycharm Build #PY-232.10300.41, I an issue with the logs not being parsed properly. Everything works fine in a regular terminal. Setting the JB_DISABLE_BUFFERING env var works but isn't ideal.

It is very easy to reproduce so I hope the fix will be easy for you. In order to reproduce, you need to run unittest fashion tests with the pytest runner and pytest-subtests installed. Then you need to setup a test class using unittest : 

from unittest import TestCase

class TestTeamCityDisplayBug(TestCase):
    def test_display(self):
        for i in range(3):
            with self.subTest(i=i):
                self.assertEqual(True, False)

When you run it you get :

============================= test session starts ==============================
collecting ... collected 1 item

adapters/driven/reporting/test_sqlalchemy_data_warehouse.py::TestTeamCityDisplayBug::test_display 
adapters/driven/reporting/test_sqlalchemy_data_warehouse.py::TestTeamCityDisplayBug::test_display PASSED [100%]

==================== 3 failed, 1 passed, 1 warning in 0.11s ====================

Process finished with exit code 1
(i=0) SUBFAIL [100%]##teamcity[testFailed timestamp='2024-02-21T20:28:00.145' actual='False' details='self = <adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug testMethod=test_display>|n|n    def test_display(self):|n        for i in range(3):|n            with self.subTest(i=i):|n>               self.assertEqual(True, False)|n|nadapters/driven/reporting/test_sqlalchemy_data_warehouse.py:31: AssertionError' expected='True' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' message='adapters/driven/reporting/test_sqlalchemy_data_warehouse.py:27 (TestTeamCityDisplayBug.test_display)|nFalse != True|n' name='test_display' nodeId='6' parentNodeId='5' type='comparisonFailure']
##teamcity[testFinished timestamp='2024-02-21T20:28:00.145' duration='0' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' name='test_display' nodeId='6' parentNodeId='5']

adapters/driven/reporting/test_sqlalchemy_data_warehouse.py::TestTeamCityDisplayBug::test_display (i=1) SUBFAIL [100%]##teamcity[testStarted timestamp='2024-02-21T20:28:00.147' captureStandardOutput='false' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' name='test_display' nodeId='7' parentNodeId='5']
##teamcity[testFailed timestamp='2024-02-21T20:28:00.147' details='self = <adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug testMethod=test_display>|n|n    def test_display(self):|n        for i in range(3):|n            with self.subTest(i=i):|n>               self.assertEqual(True, False)|n|nadapters/driven/reporting/test_sqlalchemy_data_warehouse.py:31: |n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |n/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/diff_tools.py:33: in _patched_equals|n    old(self, first, second, msg)|nE   AssertionError: True != False' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' message='adapters/driven/reporting/test_sqlalchemy_data_warehouse.py:27 (TestTeamCityDisplayBug.test_display)' name='test_display' nodeId='7' parentNodeId='5']
##teamcity[testFinished timestamp='2024-02-21T20:28:00.147' duration='0' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' name='test_display' nodeId='7' parentNodeId='5']

adapters/driven/reporting/test_sqlalchemy_data_warehouse.py::TestTeamCityDisplayBug::test_display (i=2) SUBFAIL [100%]##teamcity[testStarted timestamp='2024-02-21T20:28:00.148' captureStandardOutput='false' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' name='test_display' nodeId='8' parentNodeId='5']
##teamcity[testFailed timestamp='2024-02-21T20:28:00.148' details='self = <adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug testMethod=test_display>|n|n    def test_display(self):|n        for i in range(3):|n            with self.subTest(i=i):|n>               self.assertEqual(True, False)|n|nadapters/driven/reporting/test_sqlalchemy_data_warehouse.py:31: |n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |n/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/diff_tools.py:33: in _patched_equals|n    old(self, first, second, msg)|nE   AssertionError: True != False' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' message='adapters/driven/reporting/test_sqlalchemy_data_warehouse.py:27 (TestTeamCityDisplayBug.test_display)' name='test_display' nodeId='8' parentNodeId='5']
##teamcity[testFinished timestamp='2024-02-21T20:28:00.148' duration='0' locationHint='python</Users/philipgarnero/Workplace/intuition/jobaffinity>://adapters.driven.reporting.test_sqlalchemy_data_warehouse.TestTeamCityDisplayBug.test_display' name='test_display' nodeId='8' parentNodeId='5']
radoering commented 2 months ago

281 might fix your issue. I only paid attention to the PyCharm GUI, but as far as I understand it is based on the logs.