HigherOrderCO / Bend

A massively parallel, high-level programming language
https://higherorderco.com
Apache License 2.0
17.22k stars 425 forks source link

Whenever a test in /examples doesn't compile, the test fails instead of updating the snapshot #666

Closed In-Veritas closed 1 month ago

In-Veritas commented 1 month ago

Reproducing the behavior

Example: When running tests on the examples folder, if one doesn't compile, I get a "PoisonError", and following unrelated tests will also fail:

thread 'examples' panicked at tests/golden_tests.rs:476:51: called Result::unwrap() on an Err value: "In /home/gex/Bend/examples/list.bend :\n\u{1b}[1m- expected:\u{1b}[0m term\n\u{1b}[1m- detected:\u{1b}[0m\n\u{1b}[0m 135 | \u{1b}[4m\u{1b}[31m}\u{1b}[0m" note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- import_system stdout ---- Testing /home/gex/Bend/tests/golden_tests/import_system/import_main.bend /home/gex/Bend/tests/golden_tests/import_system/import_main.bend thread 'import_system' panicked at tests/golden_tests.rs:186:37: called Result::unwrap() on an Err value: PoisonError { .. }

---- linear_readback stdout ---- Testing /home/gex/Bend/tests/golden_tests/linear_readback/church_mul.bend /home/gex/Bend/tests/golden_tests/linear_readback/church_mul.bend thread 'linear_readback' panicked at tests/golden_tests.rs:137:35: called Result::unwrap() on an Err value: PoisonError { .. }

---- run_file stdout ---- Testing /home/gex/Bend/tests/golden_tests/run_file/360_no_scope.bend /home/gex/Bend/tests/golden_tests/run_file/360_no_scope.bend thread 'run_file' panicked at tests/golden_tests.rs:160:37: called Result::unwrap() on an Err value: PoisonError { .. }

---- cli stdout ---- Testing /home/gex/Bend/tests/golden_tests/cli/compile_all.bend /home/gex/Bend/tests/golden_tests/cli/compile_all.bend Testing /home/gex/Bend/tests/golden_tests/cli/compile_inline.bend /home/gex/Bend/tests/golden_tests/cli/compile_inline.bend thread 'cli' panicked at tests/golden_tests.rs:400:35: called Result::unwrap() on an Err value: PoisonError { .. }

System Settings

.

Additional context

No response

developedby commented 1 month ago

I think this is related to the mutex used to sequence the test which must spawn a subprocess. If more than one is ran at the same time their outputs get mangled.

If this test fails i think it fails to clear up the stdin/out properly or to maybe it releases the mutex too early.

developedby commented 1 month ago

This probably happens with other tests too, not just the examples. The difference is that the examples test fails when an example fails to compile, while the other tests store the compile error in the snapshot and so they almost never fail, only if the compiler itself crashes and similar situations.