search

craigahobbs / unittest-parallel

Parallel unit test runner for Python with coverage support
MIT License
29 stars 5 forks source link

Better output? #6

Closed wusatosi closed 3 years ago

wusatosi commented 3 years ago

With -v option, the program is spitting out a lot of chunks of 

...TESTS...
----------------------------------------------------------------------
Ran 4 tests in 3.124s

OK (skipped=4)
...TESTS...
----------------------------------------------------------------------
Ran 2 tests in 0.42s

OK (skipped=2)

This is not really helpful compared to the original unittest output which looks like this:

...TESTS...
----------------------------------------------------------------------
Ran 42 tests in 512.462s

OK (skipped=13)

Maybe a custom counting of the test runner and time took would be helpful?

craigahobbs commented 3 years ago

Thanks for the feedback. I agree that the output is chaotic. I think more closely matching the Python unittest tool's output should be fairly straightforward based on a quick look. Stay tuned.

craigahobbs commented 3 years ago

The work is mostly done in the feature/better-verbose-output branch. Give it a try and let me know what you think. You can pip-install it like so:

pip install git+https://github.com/craigahobbs/unittest-parallel.git@29d840f
wusatosi commented 3 years ago

The work is mostly done in the feature/better-verbose-output branch. Give it a try and let me know what you think. You can pip-install it like so:

pip install git+https://github.com/craigahobbs/unittest-parallel.git@29d840f

Its working flawlessly 🎉!

wusatosi commented 3 years ago

This is the log that your patch produces in my CI pipeline: Clean and verbose, great job!

test_after_grade_inquiry_deadline_no_previous_inquiry (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_no_submission_grade_inquiry_panel (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_normal_submission_grade_inquiry_panel (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_normal_submission_student_grade_inquiry_box (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_basic_info (e2e.test_my_profile.TestMyProfile) ... ok
test_edit_preferred_names (e2e.test_my_profile.TestMyProfile) ... ok
test_flagged_profile_photo (e2e.test_my_profile.TestMyProfile) ... ok
test_time_zone_selection (e2e.test_my_profile.TestMyProfile) ... ok
test_upload_profile_photo (e2e.test_my_profile.TestMyProfile) ... ok
test_instructor (e2e.test_navigation_page_non_student.TestNavigationPageNonStudent) ... ok
test_ta (e2e.test_navigation_page_non_student.TestNavigationPageNonStudent) ... ok
test_navigation_page (e2e.test_navigation_page_student.TestNavigationPageStudent) ... ok
test_office_hours_queue (e2e.test_office_hours_queue.TestOfficeHoursQueue) ... ok
test_pdf_basic_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_pdf_peer_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_pdf_peer_team_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_pdf_team_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_instructor (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_instructor_navigate_away (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_instructor_team (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_ta (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_ta_team (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_click_sidebar_links_instructor (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_instructor_blank (e2e.test_sidebar.TestSidebar) ... skipped 'cannot run in CI, blank course does not exist'
test_click_sidebar_links_instructor_sample (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_student (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_student_blank (e2e.test_sidebar.TestSidebar) ... skipped 'cannot run in CI, blank course does not exist'
test_click_sidebar_links_student_sample (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_superuser (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_ta (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_ta_blank (e2e.test_sidebar.TestSidebar) ... skipped 'cannot run in CI, blank course does not exist'
test_click_sidebar_links_ta_sample (e2e.test_sidebar.TestSidebar) ... ok
test_header_text (e2e.test_simple_grader.TestSimpleGrader) ... ok
test_basic_operations_thread (e2e.test_forum.TestForum) ... ok
test_categories (e2e.test_forum.TestForum) ... ok
test_multiple_graders (e2e.test_simple_grader.TestSimpleGrader) ... ok
test_cancel_submission_version (e2e.test_submission.TestSubmission) ... ok
test_change_submission_version (e2e.test_submission.TestSubmission) ... ok
test_forum_merge_thread (e2e.test_forum.TestForum) ... ok
test_infinite_scroll (e2e.test_forum.TestForum) ... skipped 'cannot run in CI'
test_drag_and_drop_upload (e2e.test_submission.TestSubmission) ... ok
test_drag_and_drop_upload_multiple (e2e.test_submission.TestSubmission) ... ok
test_normal_upload (e2e.test_submission.TestSubmission) ... ok
test_normal_upload_multiple (e2e.test_submission.TestSubmission) ... ok

======================================================================
test_w3_validator (e2e.test_accessibility.TestAccessibility) (url='/courses/s21/sample/office_hours_queue')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/submitty/GIT_CHECKOUT/Submitty/tests/e2e/test_accessibility.py", line 130, in validatePages
    self.assertFalse(foundErrors != [], msg=msg)
AssertionError: True is not false : 
[
    {
        "error": "Bad value 'http://localhost/courses/s21/sample/office_hours_queue/custom code/empty' for attribute 'action' on element 'form': Illegal character in path segment: space is not allowed.",
        "html extract": "<form method=\"post\" action=\"http://localhost/courses/s21/sample/office_hours_queue/custom code/empty\" onsubmit=\"return confirm('Are you sure you want to empty the queue? This will kick everyone out of the queue.');\">",
        "type": "error"
    },
    other errors...
]
More info can be found by using the w3 html validator. You can read more about it on submitty.org:
https://validator.w3.org/#validate_by_input
https://submitty.org/developer/interface_design_style_guide/web_accessibility#html-css-and-javascript

----------------------------------------------------------------------
Ran 45 tests in 323.441s

FAILED (failures=1, skipped=17)
craigahobbs commented 3 years ago

Good to hear. I need to write a few more unit tests and will release shortly.

craigahobbs commented 3 years ago

unittest-parallel 1.1.0 released