Flaky test Mina_net2.All_ipc

dannywillems commented 1 year ago

Preliminary Checks

[X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/MinaProtocol/mina/issues
[X] This issue is not a question, feature request, RFC, or anything other than a bug report. Please post those things in GitHub Discussions: https://github.com/MinaProtocol/mina/discussions

Summary

File "src/lib/mina_net2/tests/all_ipc.ml", line 642, characters 4-158: ipc test threw
--
  in TEST_MODULE at file "src/lib/mina_net2/tests/all_ipc.ml", line 64, characters 0-21706: all-ipc test

Steps to replicate this Issue

See CI

Console output

(monitor.ml.Error
  ("Mina_net2_tests.All_ipc.Timeout(\"Carol: Alice connected\")")
  ("Raised at Mina_net2_tests__All_ipc.or_timeout.(fun) in file \"src/lib/mina_net2/tests/all_ipc.ml\", line 61, characters 21-41"
    "Called from Async_kernel__Deferred1.choose.ready in file \"src/deferred1.ml\", line 137, characters 23-46"
    "Called from Async_kernel__Job_queue.run_job in file \"src/job_queue.ml\" (inlined), line 128, characters 2-5"
    "Called from Async_kernel__Job_queue.run_jobs in file \"src/job_queue.ml\", line 169, characters 6-47"
    "Caught by monitor block_on_async")).
  Raised at Base__Exn.protectx in file "src/exn.ml", line 71, characters 4-114
  Called from Ppx_inline_test_lib__Runtime.time_and_reset_random_seeds in file "runtime-lib/runtime.ml", line 356, characters 15-52
  Called from Ppx_inline_test_lib__Runtime.test in file "runtime-lib/runtime.ml", line 444, characters 52-83
  in TEST_MODULE at file "src/lib/mina_net2/tests/all_ipc.ml", line 64, characters 0-21706: all-ipc test

Platform

CI

Build link(s)

https://buildkite.com/o-1-labs-2/mina-o-1-labs/builds/30255#018a6b22-e52e-47bb-b3d5-1ac4316e1c34

deepthiskumar commented 10 months ago

on rampup- https://buildkite.com/o-1-labs-2/mina-end-to-end-nightlies/builds/863#018c3586-7e64-4a4d-90b6-9fbb676722ba

georgeee commented 5 months ago

Not sure if the issue in subject still happens (a few fixes landed targeting these tests).

But flakiness of mina_net2 tests is a real phenomenon (log from develop as of today):

machine # ======================================================================
machine # File "tests/all_ipc.ml", line 641, characters 4-158: ipc test threw
machine # (monitor.ml.Error (Mina_net2_tests.All_ipc.UnexpectedState)
machine #   ("Raised at Mina_net2_tests__All_ipc.(fun).M.carol.(fun) in file \"tests/all_ipc.ml\", line 496, characters 42-63"
machine #     "Called from Async_kernel__Deferred0.bind.(fun) in file \"src/deferred0.ml\", line 54, characters 64-69"
machine #     "Called from Async_kernel__Job_queue.run_job in file \"src/job_queue.ml\" (inlined), line 128, characters 2-5"
machine #     "Called from Async_kernel__Job_queue.run_jobs in file \"src/job_queue.ml\", line 169, characters 6-47"
machine #     "Caught by monitor block_on_async")).
machine #   Raised at Base__Exn.protectx in file "src/exn.ml", line 71, characters 4-114
machine #   Called from Ppx_inline_test_lib__Runtime.time_and_reset_random_seeds in file "runtime-lib/runtime.ml", line 356, characters 15-52
machine #   Called from Ppx_inline_test_lib__Runtime.test in file "runtime-lib/runtime.ml", line 444, characters 52-83
machine #   in TEST_MODULE at file "tests/all_ipc.ml", line 64, characters 0-21708: all-ipc test
machine # 
machine # FAILED 1 / 3 tests

MinaProtocol / mina

Flaky test Mina_net2.All_ipc #14045