Closed wusatosi closed 3 years ago
Thanks for the feedback. I agree that the output is chaotic. I think more closely matching the Python unittest tool's output should be fairly straightforward based on a quick look. Stay tuned.
The work is mostly done in the feature/better-verbose-output branch. Give it a try and let me know what you think. You can pip-install it like so:
pip install git+https://github.com/craigahobbs/unittest-parallel.git@29d840f
The work is mostly done in the feature/better-verbose-output branch. Give it a try and let me know what you think. You can pip-install it like so:
pip install git+https://github.com/craigahobbs/unittest-parallel.git@29d840f
Its working flawlessly 🎉!
This is the log that your patch produces in my CI pipeline: Clean and verbose, great job!
test_after_grade_inquiry_deadline_no_previous_inquiry (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_no_submission_grade_inquiry_panel (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_normal_submission_grade_inquiry_panel (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_normal_submission_student_grade_inquiry_box (e2e.test_grade_inquiry.TestGradeInquiry) ... skipped 'cannot run in CI'
test_basic_info (e2e.test_my_profile.TestMyProfile) ... ok
test_edit_preferred_names (e2e.test_my_profile.TestMyProfile) ... ok
test_flagged_profile_photo (e2e.test_my_profile.TestMyProfile) ... ok
test_time_zone_selection (e2e.test_my_profile.TestMyProfile) ... ok
test_upload_profile_photo (e2e.test_my_profile.TestMyProfile) ... ok
test_instructor (e2e.test_navigation_page_non_student.TestNavigationPageNonStudent) ... ok
test_ta (e2e.test_navigation_page_non_student.TestNavigationPageNonStudent) ... ok
test_navigation_page (e2e.test_navigation_page_student.TestNavigationPageStudent) ... ok
test_office_hours_queue (e2e.test_office_hours_queue.TestOfficeHoursQueue) ... ok
test_pdf_basic_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_pdf_peer_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_pdf_peer_team_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_pdf_team_access (e2e.test_pdfs.TestPDFs) ... skipped 'cannot run in CI'
test_instructor (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_instructor_navigate_away (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_instructor_team (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_ta (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_ta_team (e2e.test_prev_next.TestGradingNextPrev) ... skipped 'cannot run in CI'
test_click_sidebar_links_instructor (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_instructor_blank (e2e.test_sidebar.TestSidebar) ... skipped 'cannot run in CI, blank course does not exist'
test_click_sidebar_links_instructor_sample (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_student (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_student_blank (e2e.test_sidebar.TestSidebar) ... skipped 'cannot run in CI, blank course does not exist'
test_click_sidebar_links_student_sample (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_superuser (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_ta (e2e.test_sidebar.TestSidebar) ... ok
test_click_sidebar_links_ta_blank (e2e.test_sidebar.TestSidebar) ... skipped 'cannot run in CI, blank course does not exist'
test_click_sidebar_links_ta_sample (e2e.test_sidebar.TestSidebar) ... ok
test_header_text (e2e.test_simple_grader.TestSimpleGrader) ... ok
test_basic_operations_thread (e2e.test_forum.TestForum) ... ok
test_categories (e2e.test_forum.TestForum) ... ok
test_multiple_graders (e2e.test_simple_grader.TestSimpleGrader) ... ok
test_cancel_submission_version (e2e.test_submission.TestSubmission) ... ok
test_change_submission_version (e2e.test_submission.TestSubmission) ... ok
test_forum_merge_thread (e2e.test_forum.TestForum) ... ok
test_infinite_scroll (e2e.test_forum.TestForum) ... skipped 'cannot run in CI'
test_drag_and_drop_upload (e2e.test_submission.TestSubmission) ... ok
test_drag_and_drop_upload_multiple (e2e.test_submission.TestSubmission) ... ok
test_normal_upload (e2e.test_submission.TestSubmission) ... ok
test_normal_upload_multiple (e2e.test_submission.TestSubmission) ... ok
======================================================================
test_w3_validator (e2e.test_accessibility.TestAccessibility) (url='/courses/s21/sample/office_hours_queue')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/submitty/GIT_CHECKOUT/Submitty/tests/e2e/test_accessibility.py", line 130, in validatePages
self.assertFalse(foundErrors != [], msg=msg)
AssertionError: True is not false :
[
{
"error": "Bad value 'http://localhost/courses/s21/sample/office_hours_queue/custom code/empty' for attribute 'action' on element 'form': Illegal character in path segment: space is not allowed.",
"html extract": "<form method=\"post\" action=\"http://localhost/courses/s21/sample/office_hours_queue/custom code/empty\" onsubmit=\"return confirm('Are you sure you want to empty the queue? This will kick everyone out of the queue.');\">",
"type": "error"
},
other errors...
]
More info can be found by using the w3 html validator. You can read more about it on submitty.org:
https://validator.w3.org/#validate_by_input
https://submitty.org/developer/interface_design_style_guide/web_accessibility#html-css-and-javascript
----------------------------------------------------------------------
Ran 45 tests in 323.441s
FAILED (failures=1, skipped=17)
Good to hear. I need to write a few more unit tests and will release shortly.
unittest-parallel 1.1.0 released
With
-v
option, the program is spitting out a lot of chunks ofThis is not really helpful compared to the original unittest output which looks like this:
Maybe a custom counting of the test runner and time took would be helpful?