Apart from the timeouts, there're also exceptions like this:
2024-06-10 13:21:33.481402Z Error ("Exception raised to [Monitor.try_with] that already returned.""This error was captured by a default handler in [Async.Log]."(exn(monitor.ml.Error(runtime-lib/runtime.ml.E"comparison failed"(9628362212128880070336703606391899380685046295904155096315041590933245033541 vs 16391248562513897626799161937302081616700277556236632679730397125656290430570(Loc src/lib/transaction_snark/test/util.ml:282:33)))("Raised at Ppx_assert_lib__Runtime.failwith in file \"runtime-lib/runtime.ml\", line 28, characters 28-53""Called from Async_kernel__Deferred1.M.map.(fun) in file \"src/deferred1.ml\", line 17, characters 40-45""Called from Async_kernel__Job_queue.run_job in file \"src/job_queue.ml\" (inlined), line 128, characters 2-5""Called from Async_kernel__Job_queue.run_jobs in file \"src/job_queue.ml\", line 169, characters 6-47"))))
Investigation by Jiawei:
This buildkite run seems very weird, a lot of individual tests took more than 5 mins and some of tests took 9 mins. This is very unusual. I failed to reproduce this locally after 4 tries.
I assume this is caused by limited resource allocated to this test.
Fuzzy zkapp test failed: https://buildkite.com/o-1-labs-2/mina-end-to-end-nightlies/builds/2102#0190021d-2bb9-4231-9315-57eda45e2295/111-499
Apart from the timeouts, there're also exceptions like this:
Investigation by Jiawei: This buildkite run seems very weird, a lot of individual tests took more than 5 mins and some of tests took 9 mins. This is very unusual. I failed to reproduce this locally after 4 tries.
I assume this is caused by limited resource allocated to this test.