When passing additional arguments in the test suite, they are added to the function call as arg1, arg2, etc.
The context is a class with the following fields:
class ConvertedOracleContext:
expected: Any # Expected value from the test suite
actual: Any # Value generated by the submission
execution_directory: str # Directory where the execution took place (this will be a subdirectory of the workdir)
evaluation_directory: str # Path to the evaluation directory (in the exercise directory)
programming_language: str # Name of the programming language of the submission
natural_language: str # Natural language
This replaces the expected/actual values and provides some more context for the oracles.
Fixes #472 and closes #470.
A check function must now look like this:
When passing additional arguments in the test suite, they are added to the function call as
arg1
,arg2
, etc.The context is a class with the following fields: