Beluga-lang / Beluga

Contextual types meet mechanized metatheory!
http://complogic.cs.mcgill.ca/beluga/
GNU General Public License v3.0
184 stars 16 forks source link

Broken interactive mode tests #264

Closed MartyO256 closed 1 year ago

MartyO256 commented 1 year ago

The test runner for the interactive mode tests in TEST.sh has been broken since commit b66d864fcf81c0de6ed78b811044e5b8a922c3cb. This is because in Bash, retrieving the exit code of a program run in local output=program with exit_code=$? will actually always return 0 as the exit code of local (c.f. https://stackoverflow.com/a/4421282).

As a consequence, the interactive mode tests have always been reported as succeeding, and have not been properly maintained since b66d864fcf81c0de6ed78b811044e5b8a922c3cb. The branch fix-test fixes the test runner, but not the interactive tests themselves.

Right now, 15 of the 16 interactive mode tests are failling silently, even though the test case ./t/interactive/logic-programming/1.bel has been failing since the recent changes to the proof search engine.

===== INTERACTIVE MODE TESTS =====
** TEST 435: ./t/interactive/44.bel ... OK
** TEST 436: ./t/interactive/46.bel ... OK
** TEST 437: ./t/interactive/82.bel ... OK
** TEST 438: ./t/interactive/96.bel ... OK
Uncaught exception.
Please report this as a bug.
** TEST 439: ./t/interactive/logic-programming/1.bel ... OK