alephcloud / hs-yet-another-logger

Haskell logging framework with a focus on flexibility and performance
Other
5 stars 4 forks source link

yet-another-logger-0.2.3.1 test suite failure #38

Open peti opened 9 years ago

peti commented 9 years ago

Citing from http://hydra.cryp.to/build/903657/log/raw. The test suite says that all test have succeeded, and then fails:

Running 1 test suites...
Test suite tests: RUNNING...
trivial backend
  no backend
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 1}:                OK (105.24s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 1}:             OK (106.71s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 1}:         OK (106.05s)
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 1}:                OK (105.30s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 1}:             OK (107.02s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 1}:         OK (106.52s)
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 1}:                 OK (105.17s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 1}:              OK (107.61s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 1}:          OK (113.44s)
  no backend
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 100}:              OK (111.89s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 100}:           OK (113.27s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 100}:       OK (113.46s)
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 100}:              OK (111.49s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 100}:           OK (112.12s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 100}:       OK (112.75s)
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 100}:               OK (110.50s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 100}:            OK (121.49s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 100}:        OK (120.81s)
  no backend
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 100000000}:        OK (121.95s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 100000000}:     OK (202.85s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 100000000}: OK (204.47s)
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 100000000}:        OK (121.73s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 100000000}:     OK (201.75s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 100000000}: OK (202.86s)
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 100000000}:         OK (73.73s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 100000000}:      OK (82.59s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 1, backendDelay = 5, exitDelay = Just 100000000}:  OK (83.08s)
  buggy backend 11 10
    TestParams {queueSize = 10, threadsN = 10, messageN = 100, messageSize = 1000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 10}:                OK (0.97s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 10}:            OK (0.94s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 10000, frontendDelay = 25, backendDelay = 1, exitDelay = Just 10}:        OK (0.68s)
  buggy recover backend 2 10
    TestParams {queueSize = 10, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 100, backendDelay = 1, exitDelay = Just 10}:              OK (70.09s)
    TestParams {queueSize = 1000, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 10}:             OK (8.21s)
    TestParams {queueSize = 10000000, threadsN = 100, messageN = 100, messageSize = 1000, frontendDelay = 10, backendDelay = 1, exitDelay = Just 10}:         OK (7.64s)
  buggy recover backend 8 10
    TestParams {queueSize = 1000, threadsN = 10, messageN = 100, messageSize = 100, frontendDelay = 100, backendDelay = 1, exitDelay = Just 10}:              OK (0.57s)
  buggy no recover backend 10 15
    TestParams {queueSize = 1000, threadsN = 10, messageN = 100, messageSize = 100, frontendDelay = 100, backendDelay = 1, exitDelay = Just 1000}:            OK (0.14s)

All 35 tests passed (423.41s)
Test suite tests: FAIL
peti commented 9 years ago

Is there a chance that this issue can be fixed in the foreseeable future? It's kind of annoying for our build server if this package fails every couple of attempts to compile it, and I'm tempted to just remove it from the distribution because it pops up in the failed builds so frequently.

larskuhtz commented 9 years ago

@peti I agree that it is annoying. It seems to happen randomly. I guess that it is related to some exception escaping its scope and popping up after the test-suite logged all tests as successful. It's most likely a test artifact, since in the tests we try to trigger some uncommon exceptional scenarios.

I am currently somewhat busy so I haven't had yet a chance to look at it. I'll try to get to it during this or the next week.

peti commented 9 years ago

OK. I've disable the test builds runs on our server for the time being. Thank you for the quick response!