sourcegraph / sourcegraph-public-snapshot

Code AI platform with Code Search & Cody
https://sourcegraph.com
Other
10.12k stars 1.29k forks source link

customer-replica branch of pure-docker test expects incorrect number of containers #39321

Open coury-clark opened 2 years ago

coury-clark commented 2 years ago

In the 3.42 release we encountered an issue where the smoke tests for the customer replica branch of pure_docker were failing. There was a change since the 3.41 release that changed the expected containers from 61 to 62. In CI and running locally the result was 60 containers.

In the 3.41-customer-replica the tests passed against 61 containers.

It isn't clear why the discrepancy, but the above PR does make a note that there was to be follow up work that appears not to be complete.

Failing 3.42 build: https://buildkite.com/sourcegraph/deploy-sourcegraph-docker/builds/11853 Passing 3.41 build: https://buildkite.com/sourcegraph/deploy-sourcegraph-docker/builds/11362

Because this issue appears to be an issue with the test, we set the expected count to 60 to proceed with the 3.42 release.

sourcegraph-bot-2 commented 2 years ago

Heads up @sourcegraph/delivery - the "team/delivery" label was applied to this issue.

coury-clark commented 2 years ago

After some more trial and error, @davejrt noticed that there was only a 30 second timeout to allow the containers to start. It seems by changing these two values we were able to get the tests passing:

  1. 62 -> 60 expected containers
  2. 30 -> 120 second timeout