search

StyraInc / regal

Regal is a linter and language server for Rego, bringing your policy development experience to the next level!
https://docs.styra.com/regal
Apache License 2.0
261 stars 35 forks source link

Fix flaky TestLanguageServerMultipleFiles #931

Closed anderseknert closed 3 months ago

anderseknert commented 3 months ago

This is annoying, and a bad contributor experience, as it's nothing they've caused or can address. We should fix this ASAP.

   --- FAIL: TestLanguageServerMultipleFiles (4.01s)
      server_test.go:406: timed out waiting for authz.rego diagnostics to be sent
  ERROR: failed to update aggregate diagnostics (trigger): failed to lint: failed to lint using Rego aggregate rules: error encountered in query evaluation eval_cancel_error: caller cancelled query execution
  ERROR: failed to send diagnostic: failed to notify: jsonrpc2: connection is closed
  ERROR: failed to send diagnostic: failed to notify: jsonrpc2: connection is closed
  FAIL
charlieegan3 commented 3 months ago

I've done some digging here, it looks to me that this is (now, after some previous adjustments to this test) just a case of the timeouts being too aggressive to be reliable on GH actions.

I have been doing some testing today and found that in 300 runs the max time is around 1.8s to wait for a file's state to be consistent with the expected value.

I think that the fix here is going to be making the timeouts longer for now as I've not been able to replicate any locking behaviour that causes this test to fail in another wait other than the timeout.

I have increased the timeout in #933