We recently added an integration test that starts a total of 80 dev server, with a maximum of 8 instances running concurrently. This is used to assert, among other things, the reliability of the "get ephemeral port number" logic.
However, that test is incorrect, because it calls FailNow and similar from goroutines, which is not allowed. The test is also flaky. Incorrect usage of test APIs is like not the cause of the flakiness, but may be masking the actual errors, making the issue harder to diagnose.
Given that the CLI's test harness internally use FailNow, fixing this will likely require developing a substitute for the existing test harness.
Describe the bug
We recently added an integration test that starts a total of 80 dev server, with a maximum of 8 instances running concurrently. This is used to assert, among other things, the reliability of the "get ephemeral port number" logic.
However, that test is incorrect, because it calls
FailNowand similar from goroutines, which is not allowed. The test is also flaky. Incorrect usage of test APIs is like not the cause of the flakiness, but may be masking the actual errors, making the issue harder to diagnose.Given that the CLI's test harness internally use
FailNow, fixing this will likely require developing a substitute for the existing test harness.