search

ualbertalib / jupiter

Jupiter is a University of Alberta Libraries-based initiative to create a sustainable and extensible digital asset management system. This is phase 2 (Digitization).
https://era.library.ualberta.ca/
MIT License
23 stars 10 forks source link

Occasional Test Suite failures #2129

Open pgwillia opened 3 years ago

pgwillia commented 3 years ago

Describe the bug Occasionally CI will catch failing tests like the following:

Failure:
DoiResetTest#test_reset_button_displays_as_needed [/home/runner/work/jupiter/jupiter/test/integration/doi_reset_test.rb:19]:
Expected exactly 0 elements matching ".admin-sidebar .btn", found 1..
Expected: 0
  Actual: 1

rails test test/integration/doi_reset_test.rb:12

To Reproduce Run options: --seed 41156, --seed 29156, --seed 42894, --seed 19850, --seed 31485, --seed 32895, --seed 21699, --seed 9667

Expected behavior Stable and consistent test suite.

Additional context There are others I've seen recently. I'll update as I experience them.

pgwillia commented 3 years ago 
Failure:
ReadOnlyModeIntegrationTest#test_read_only_mode_creates_and_clears_announcement [/Users/matt/source/ualib/jupiter/test/integration/read_only_mode_integration_test.rb:30]:
Expected exactly 1 element matching "div.alert", found 2..
Expected: 1
  Actual: 2
pgwillia commented 3 years ago

rails system:test on WSL2 (so might just be my machine 😅 )

Failure:
AdminUsersShowTest#test_should_be_able_to_login_as_a_regular_user [/mnt/c/Users/pgwillia/Code/ualbertalib/jupiter/test/system/admin_users_show_test.rb:116]:
expected to find visible css "h1" with text "Dashboard" but there were no matches. Also found "ERA:\nEducation and Research Archive", which matched the selector but not all filters.

rails test test/system/admin_users_show_test.rb:107
Error:
EditItemOldLicenseTest#test_can_edit_and_save_an_item_with_an_old_license:
Capybara::ElementNotFound: Unable to find link or button "Save and continue to next step"
    test/system/edit_item_old_license_test.rb:21:in `block in <class:EditItemOldLicenseTest>'

rails test test/system/edit_item_old_license_test.rb:5

failures_test_can_edit_and_save_an_item_with_an_old_license

Failure:
AdminUsersIndexTest#test_should_be_able_to_sort_columns [/mnt/c/Users/pgwillia/Code/ualbertalib/jupiter/test/system/admin_users_index_test.rb:25]:
expected to find visible css "tbody tr:last-child th[scope=\"row\"]" with text "administrator@example.com" but there were no matches. Also found "tyrion_lannister@example.com", which matched the selector but not all filters.

rails test test/system/admin_users_index_test.rb:5

failures_test_should_be_able_to_sort_columns

pgwillia commented 3 years ago 
Failure:
AdminUsersIndexTest#test_should_be_able_to_autocomplete_by_name [/home/runner/work/jupiter/jupiter/test/system/admin_users_index_test.rb:56]:
expected to find visible css "tbody tr" 1 time, found 6 matches: "administrator@example.com Administrator 2021-04-23 less than a minute ago Admin Active", "tyrion_lannister@example.com Tyrion Lannister 2021-04-23 Never Signed In Admin Active", "john_snow@example.com John Snow 2021-04-23 Never Signed In User Active", "daenerys_targaryen@example.com Daenerys Targaryen 2021-04-23 Never Signed In User Active", "joffrey_baratheon@example.com Joffrey Baratheon 2021-04-23 Never Signed In User Suspended", "ditech@ualberta.ca System user 2021-04-23 Never Signed In User Active"

rails test test/system/admin_users_index_test.rb:46

--seed 22431, 52002, 36603, 4214, 56167, 38933, 24566

pgwillia commented 3 years ago 
Failure:
DepositThesisTest#test_be_able_to_deposit_and_edit_a_thesis_successfully [/home/runner/work/jupiter/jupiter/test/system/deposit_thesis_test.rb:62]:
expected to find visible css "h2" with text "Review and Deposit Thesis" but there were no matches. Also found "Upload Files", which matched the selector but not all filters. 

rails test test/system/deposit_thesis_test.rb:5

--seed 26145, 10422, 22708, 12310, 3059

pgwillia commented 3 years ago 
Error:
AdminUsersIndexTest#test_should_be_able_to_filter_by_status:
Capybara::ElementNotFound: Unable to find link "Users"
    test/system/admin_users_index_test.rb:69:in `block in <class:AdminUsersIndexTest>'

rails test test/system/admin_users_index_test.rb:62

--seed 5723

pgwillia commented 3 years ago 
Failure:
DepositItemTest#test_be_able_to_deposit_and_edit_an_item_successfully [/home/runner/work/jupiter/jupiter/test/system/deposit_item_test.rb:65]:
expected to find visible css "h2" with text "Review and Deposit Item" but there were no matches. Also found "Upload Files", which matched the selector but not all filters. 

rails test test/system/deposit_item_test.rb:5

--seed 4532, 15598, 25949, 22270, 5261, 1858

Failure:
DepositThesisTest#test_be_able_to_deposit_and_edit_a_thesis_successfully [/home/runner/work/jupiter/jupiter/test/system/deposit_thesis_test.rb:62]:
expected to find visible css "h2" with text "Review and Deposit Thesis" but there were no matches. Also found "Upload Files", which matched the selector but not all filters. 

rails test test/system/deposit_thesis_test.rb:5

--seed 57789, 27940

Error:
DepositItemTest#test_be_able_to_deposit_and_edit_an_item_successfully:
Selenium::WebDriver::Error::StaleElementReferenceError: stale element reference: element is not attached to the page document
  (Session info: headless chrome=92.0.4515.131)
    test/system/deposit_item_test.rb:142:in `block in selectize_set_text'
    test/system/deposit_item_test.rb:140:in `selectize_set_text'
    test/system/deposit_item_test.rb:27:in `block in <class:DepositItemTest>'

rails test test/system/deposit_item_test.rb:5

--seed 7303, 52143, 52921, 42387, 12889

Error:
DepositItemTest#test_be_able_to_deposit_and_edit_an_item_successfully:
Selenium::WebDriver::Error::StaleElementReferenceError: stale element reference: element is not attached to the page document
  (Session info: headless chrome=94.0.4606.81)
    test/system/deposit_item_test.rb:142:in `block in selectize_set_text'
    test/system/deposit_item_test.rb:140:in `selectize_set_text'
    test/system/deposit_item_test.rb:25:in `block in <class:DepositItemTest>'

rails test test/system/deposit_item_test.rb:5

--seed 1837

pgwillia commented 3 years ago

I thought I had fixed this by adding a selector that would wait for the change. Now that selector is failing too.

Failure:
AdminUsersIndexTest#test_should_be_able_to_autocomplete_by_name [/home/runner/work/jupiter/jupiter/test/system/admin_users_index_test.rb:58]:
expected to find visible css "div" with text "Displaying 1 of 1 matching users" but there were no matches. Also found "Dashboard\nUsers\nAnnouncements\nCommunities and Collections\nItems\nFeature Flags\nUsers\nSearch\nRole\nAll\nAdmin\nUser\nStatus\nAll\nSuspended\nActive\nEmail  Name  Registered  Last Seen  Role  Status \nadministrator@example.com Administrator 2021-08-25 less than a minute ago Admin Active\ntyrion_lannister@example.com Tyrion Lannister 2021-08-25 Never Signed In Admin Active\njohn_snow@example.com John Snow 2021-08-25 Never Signed In User Active\ndaenerys_targaryen@example.com Daenerys Targaryen 2021-08-25 Never Signed In User Active\njoffrey_baratheon@example.com Joffrey Baratheon 2021-08-25 Never Signed In User Suspended\nditech@ualberta.ca System user 2021-08-25 Never Signed In User Active\nDisplaying 6 of 6 matching users (6 total users)", "Email  Name  Registered  Last Seen  Role  Status \nadministrator@example.com Administrator 2021-08-25 less than a minute ago Admin Active\ntyrion_lannister@example.com Tyrion Lannister 2021-08-25 Never Signed In Admin Active\njohn_snow@example.com John Snow 2021-08-25 Never Signed In User Active\ndaenerys_targaryen@example.com Daenerys Targaryen 2021-08-25 Never Signed In User Active\njoffrey_baratheon@example.com Joffrey Baratheon 2021-08-25 Never Signed In User Suspended\nditech@ualberta.ca System user 2021-08-25 Never Signed In User Active\nDisplaying 6 of 6 matching users (6 total users)", which matched the selector but not all filters. 

rails test test/system/admin_users_index_test.rb:48

--seed 35483, 48410, 7466, 28126, 7344