codecov / codecov-cli

Codecov's Command Line Interface. Used for uploading to Codecov in your CI, Test Labelling, Local Upload, and more
Apache License 2.0
56 stars 39 forks source link

Aggregate the test results files in CI #507

Closed michelletran-codecov closed 2 months ago

michelletran-codecov commented 2 months ago

Update the CI to aggregate results all into one comment. This will be useful for extracting into its own action in the future.

Basically, changes needed to ensure one comment:

I've added a breaking test for testing purposes. Once I get an approval, I'll remove it.

codecov[bot] commented 2 months ago

:x: 5 Tests Failed:

Tests completed Failed Passed Skipped
3485 5 3480 0
View the top 3 failed tests by shortest run time > > ``` > api.temp.calculator.test_calculator test_divide > ``` > >
Stack Traces | 0.001s run time > > > > > ``` > > def > > test_divide(): > > > assert Calculator.divide(1, 2) == 0.5 > > E assert 1.0 == 0.5 > > E + where 1.0 = <function Calculator.divide at 0x104c9eb90>(1, 2) > > E + where <function Calculator.divide at 0x104c9eb90> = Calculator.divide > > .../temp/calculator/test_calculator.py:30: AssertionError > > ``` > >
api.temp.calculator.test_calculator test_divide
Stack Traces | 0.001s run time > > ``` > def > test_divide(): > > assert Calculator.divide(1, 2) == 0.5 > E assert 1.0 == 0.5 > E + where 1.0 = <function Calculator.divide at 0x104c9eb90>(1, 2) > E + where <function Calculator.divide at 0x104c9eb90> = Calculator.divide > .../temp/calculator/test_calculator.py:30: AssertionError > ```
api.temp.calculator.test_calculator test_divide
Stack Traces | 0.001s run time > > ``` > def > test_divide(): > > assert Calculator.divide(1, 2) == 0.5 > E assert 1.0 == 0.5 > E + where 1.0 = <function Calculator.divide at 0x104c9eb90>(1, 2) > E + where <function Calculator.divide at 0x104c9eb90> = Calculator.divide > .../temp/calculator/test_calculator.py:30: AssertionError > ```

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

github-actions[bot] commented 2 months ago

:x: Failed Test Results:

Completed 3470 tests with 5 failed, 3465 passed and 0 skipped.

View the full list of failed tests | **Test Description** | **Failure message** | | :-- | :-- | |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
|
github-actions[bot] commented 2 months ago

:x: Failed Test Results:

Completed 3470 tests with 5 failed, 3465 passed and 0 skipped.

View the full list of failed tests | **Test Description** | **Failure message** | | :-- | :-- | |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
| |
Testsuite:
tests.commands.test_process_test_results::test_fail

Test name:
pytest
|
def test_fail():
&gt; assert False
E assert False

tests/commands/test_process_test_results.py:10: AssertionError
|
github-actions[bot] commented 2 months ago

:x: Failed Test Results:

Completed 3465 tests with 0 failed, 3465 passed and 0 skipped.

View the full list of failed tests | **Test Description** | **Failure message** | | :-- | :-- |
github-actions[bot] commented 2 months ago

:x: Failed Test Results:

Completed 3465 tests with 0 failed, 3465 passed and 0 skipped.

View the full list of failed tests | **Test Description** | **Failure message** | | :-- | :-- |