Better CI setup for simple understanding of failed tests - Githubissues

rsocket / rsocket-kotlin

RSocket Kotlin multi-platform implementation

http://rsocket.io

Apache License 2.0

553 stars 37 forks source link

Better CI setup for simple understanding of failed tests #147

Closed whyoleg closed 3 years ago

whyoleg commented 3 years ago

Motivation:

On current moment, it's hard to understand which test, for which target and why it failed.

Modifications:

split running tests per target
integrate mikepenz/action-junit-report github action to show report about failed tests and stacktraces
run jvm tests on 8, 11 and 15 JVM versions
split tests in 3 groups: core, local, ktor
added timeouts for tests - sometimes ktor K/N tests are just freeze - still WIP why
turn on debug logs of test runner on CI - to understand tests freezing
better exception if test fails in before/after

Result:

Easier to understand, what to fix. Still, CI tests are unstable (locally jvm/js/mac tests works ok)

whyoleg commented 3 years ago

UPDATE:

removed Tomcat websocket test - it's slow, ktor should have sufficient tests to make sure, that engines WS implementation is the same. CIO/Netty/Jetty left, as they are anyway fast and popular.
ignored 3 long tests on K/N to speed up tests - now all tests are executed in less than 20 minutes
removed gradle-all workflow - when doing PR it creates runs double of jobs needed. If needed to run tests for specific branch - Run tests workflow can be used from Github UI.