search

seas-computing / course-planner

SEAS Course Planning Application. JSDoc Docs at: https://seas-computing.github.io/course-planner/
0 stars 1 forks source link

Investigate phantom test failures #628

Open jonseitz opened 1 year ago

jonseitz commented 1 year ago

We continue to see frequent failing test runs, which usually resolve after re-running the test suite a few times. It's almost always a single end-to-end test, though it's not consistently the same one. I thought that #521 and #522 would have fixed it, but the issue still seems to be there. As a course for investigation, I'd like to:

  1. Look into most recent unexplainable test failure, and try to triangulate which tests are failing.
  2. Look into those specific failing tests, as well as the ones around them in case of aysnc leakages.
  3. Watch test runs over the next week to check for continued failures.
  4. For future, look into how we might increase performance on test runs, to close up our iteration cycles.
jonseitz commented 1 year ago

Error on 2023-01-17 16:07:

1784 passing (17m)
  1 failing

  1) End-to-end Course Instance updating
       As an admin user
         Updating Offered Values
           Editing an offered value
             to "Retired"
               for a future course instance
                 "before each" hook for "should update the offered value of future course instances to be OFFERED.RETIRED":
     Error: Timeout of 30000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/course-planner/course-planner/tests/integration/e2e/CoursesPage.test.tsx)
      at listOnTimeout (internal/timers.js:557:17)
      at processTimers (internal/timers.js:500:7)
jonseitz commented 1 year ago

FYI @rmainseas, I pushed a couple of small changes on the bugfix/failing-builds branch that address some of the issues that I was seeing when running the tests locally. There are still tests failing in the Github Actions, but I'm seeing fewer errors at least.