freelawproject / courtlistener

A fully-searchable and accessible archive of court data including growing repositories of opinions, oral arguments, judges, judicial financial records, and federal filings.
https://www.courtlistener.com
Other
548 stars 150 forks source link

Flaky tests: `test_multiple_alerts_email_hits_limit_per_alert` #4624

Open cweider opened 3 weeks ago

cweider commented 3 weeks ago

With #4596 merged, GitHub scrolls to the test failure logs (instead of selenium screenshot step) and apparent that 1-2 tests are accounting for sporadic errors. I’d estimate this is happening ~5%:

FAIL: test_multiple_alerts_email_hits_limit_per_alert (cl.alerts.tests.tests_recap_alerts.RECAPAlertsSweepIndexTest.test_multiple_alerts_email_hits_limit_per_alert)
AssertionError: 2 != 0 : Did not get the right number of child hits for the case SUBPOENAS SERVED CASE. Expected: 0 - Got: 2
FAIL: test_party_api_query_counts (cl.api.tests.ApiQueryCountTests.test_party_api_query_counts)
AssertionError: 0 != 9 : 0 queries executed, 9 expected

Example runs:

cweider commented 3 weeks ago

It is doubtful that I’m in any position to address the failures – only filing the ticket.

mlissner commented 3 weeks ago

Thanks. We'll get it on the list.

mlissner commented 4 days ago

This one is now assigned and set for the current sprint. Please let me know if I got the size wrong. :)

ERosendo commented 1 day ago

I submitted two pull requests to address the reported issues: