ds26gte
commented
6 months ago - runExamplarAsync() rewritten to run all chaffs only if wheats exist and all succeed. If any wheat fails, and if hints.json exists, run chaffs specified in the latter until one succeeds -- and pass its associated hint to handleRunExamplarSuccessFull().
- ExamplarReport, ExamplarReportProps have field hintMessage.
- handleRunExamplarSuccess(), handleRunExamplarSuccessFull() take hintMessage arg
- resultSummary(): tighten messages (no need to talk of chaffs if any wheat failed). Less personal ('your tests' rather than 'you').
- Added Quartermaster feature:
- runExamplarAsync(): use only the non-QtM parts of a check block's results to decide if tests passed
- QtM clauses are checked to calculate number of QtM variants that got checked
- accumulate QtM variations across all chaffs. QtM checks are expected to be true for a chaff/wheat to pass.
- add QtM variations by (re)defining QtM predicate inside a chaff that raises an exception for an acceptable arg!
- pass along number of QtM variants to all the objects & functions handling results
- resultSummary(): use the number of QtM variants to tack on QtM diagnostic
- added subroutines stringPart(), addCodeToCheckArray(), getQtmVariations() for recognizing check-block clauses, for extracting code fragments, & calculating QtM variations tested
- Mandate reference/user implementation in the top (projects/) dir to have filename ending in -code.arr. This is the file in test.arr that's replaced by a chaff/wheat file. This makes it more robust than assuming ref impl is the first file(...) in test.arr
- lists.arr: uncripple shuffle(), we need it for some assignments