venasolutions / bosk

Apache License 2.0
11 stars 4 forks source link

False-success in automated tests #104

Closed prdoyle closed 9 months ago

prdoyle commented 9 months ago

There's a case where the tests run very quickly and report a passing result, which seems suspicious.

I used the GitHub UI to resolve a merge conflict. That created a merge commit ee1c8bfc657dd34fa427c7d57095499e73598a44, and kicked off the automated tests here. Those tests "passed" in a little over 2 minutes, when they ought to take over 10 minutes. The tests on the prior commit had not yet finished. Screen shot below.

I imagine I'm benefiting from Gradle caching, and Gradle somehow determined I didn't need to re-run the tests. This seems bad. I worry that I have my Gradle setup wrong somehow.

image

prdoyle commented 9 months ago

Duh. Figured it out here.