WikiEducationFoundation / WikiEduDashboard

Wiki Education Foundation's Wikipedia course dashboard system
https://dashboard.wikiedu.org
MIT License
392 stars 630 forks source link

assigned_articles_spec.rb is flaky #5652

Open ragesoss opened 9 months ago

ragesoss commented 9 months ago

What is happening?

The 'Assigned Articles' feature spec fails intermittently on CI. For example: https://github.com/WikiEducationFoundation/WikiEduDashboard/actions/runs/7889907818/job/21530874633

  1) Assigned Articles view lets users submit feedback about articles
     Got 0 failures and 2 other errors:

     1.1) Failure/Error: find('textarea.feedback-form').fill_in with: 'This is a great article!'

          Capybara::ElementNotFound:
            Unable to find css "textarea.feedback-form"
          # ./spec/features/assigned_articles_spec.rb:26:in `block (3 levels) in <top (required)>'
          # ./spec/features/assigned_articles_spec.rb:22:in `block (2 levels) in <top (required)>'

     1.2) Failure/Error: Capybara::Screenshot.new.screenshot_and_save_page if example.exception

          NoMethodError:
            undefined method `new' for Capybara::Screenshot:Module

                Capybara::Screenshot.new.screenshot_and_save_page if example.exception
                                    ^^^^
          # ./spec/rails_helper.rb:101:in `block (2 levels) in <top (required)>'

Update the test to pass reliably, and describe what made the test unreliable if you can.

It may be useful to remove the --headless=new parameter from the Capybara config in rails_helper.rb so that you can watch the test run in the browser. (Be sure not to include that change to rails_helper in the PR.)

PR4NJ41 commented 9 months ago

@ragesoss I have raised a PR to fix this. I fixed Error 1.1 & afterwards 1.2 stopped showing automatically. But i suppose error due to Capybara still persist

ragesoss commented 9 months ago

Yes, I opened a separate issue for the Capybara screenshot error.

ragesoss commented 9 months ago

This seems related to too-slow network requests that rely on external services. I think the right fix is probably to identify the slow request, and then stub it (with a new method in request_helpers.rb) so that the test can run quickly without actually hitting the slow API.