rancher / distros-test-framework

5 stars 9 forks source link

Improve timeouts #84

Open rancher-max opened 7 months ago

rancher-max commented 7 months ago

Coming from https://github.com/rancher/distros-test-framework/pull/74/files#r1540032497 The first job timed out after an hour. The second job completed successfully after 9 minutes. The only difference between the two was a sleep 20. This isn't the only time this has happened, but this is a good case for this.

We need to improve timeouts to:

  1. Have shorter times. If in fact 20 seconds is enough in this case, then that test/function/etc. should timeout after 20 seconds, not an hour.
  2. Give better information; show what/where/why it is timing out. The timed out test failure here doesn't give any useful information. It just shows the test itself that was running, not where in the test exactly it timed out and why (see below).

    panic: test timed out after 1h0m0s
    running tests:
    TestVersionBumpSuite (1h0m0s)
    
    goroutine 195 [running]:
    testing.(*M).startAlarm.func1()
    /usr/local/go/src/testing/testing.go:2259 +0x3b9
    created by time.goFunc
    /usr/local/go/src/time/sleep.go:176 +0x2d
    
    goroutine 1 [chan receive, 60 minutes]:
    testing.(*T).Run(0xc000202340, {0x1151785?, 0x531d3c?}, 0x11c7390)
    /usr/local/go/src/testing/testing.go:1649 +0x3c8
    testing.runTests.func1(0x1e148a0?)
    /usr/local/go/src/testing/testing.go:2054 +0x3e
    testing.tRunner(0xc000202340, 0xc00057fba0)
    /usr/local/go/src/testing/testing.go:1595 +0xff
    testing.runTests(0xc000174fa0?, {0x1dda6e0, 0x1, 0x1}, {0x1e148a0?, 0xc0000defc8?, 0x1e139c0?})
    /usr/local/go/src/testing/testing.go:2052 +0x445
    testing.(*M).Run(0xc000174fa0)
    /usr/local/go/src/testing/testing.go:1925 +0x636
    github.com/rancher/distros-test-framework/entrypoint/versionbump.TestMain(0x472fba?)
    /go/src/github.com/rancher/distros-test-framework/entrypoint/versionbump/versionbump_suite_test.go:60 +0x545
    main.main()
    _testmain.go:49 +0x1c6
fmoral2 commented 6 months ago

this shouldnt be a design topic ? @rancher-max

rancher-max commented 6 months ago

It may or may not need to be... this one will have to start with some analysis and probably be split up into sections to cover each test