search

smartystreets / goconvey

Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.
http://smartystreets.github.io/goconvey/
Other
8.24k stars 555 forks source link

fix(web): fix nondeterministic tests #688

Closed srabraham closed 7 months ago

srabraham commented 8 months ago

These integration tests were very flaky before. Running them repeatedly, e.g. by while true; do go test -count=1 ./web/server/watch; done, would lead to different errors each time on assertions about watcher results outputs.

I traced this back to integration.go's Watcher.Listen(). This function is designed to loop forever, each time looking for changes in the local filesystem, then responding if it does find changes. It's all really janky, to be honest. The problem here was that the tests were very sensitive to when input commands came in. If the recipient got two commands quickly, then you'd have the two commands processed before a scan could happen (in the select's default case). With this change, the select will always do a scan after each command is run, and it also won't sleep unnecessarily while a command is arriving.

This required an update to one test, because this caused it to run scan twice more, which is innocuous and fine.

I think this code is used for filewatching for the test web UI, and I confirmed locally that it still works.

srabraham commented 8 months ago

FYI @riannucci. This makes it so that go test ./... totally passes for me, which is awesome

riannucci commented 7 months ago

LG w/ question re: test