This PR adds error raising if any metric is missing an argument from task output dictionary:
opik.exceptions.ScoreMethodMissingArguments: The scoring object is_json_metric is missing arguments: ['output']. These keys were not present in the dictionary returned by the evaluation task. Evaluation task dictionary keys found: ['model_output', 'input', 'context'].
Issues
Resolves an issue when it was not clear what's wrong if someone didn't add required key-value pair to task output dictionary because SDK was hiding this error.
Details
This PR adds error raising if any metric is missing an argument from task output dictionary:
opik.exceptions.ScoreMethodMissingArguments: The scoring object is_json_metric is missing arguments: ['output']. These keys were not present in the dictionary returned by the evaluation task. Evaluation task dictionary keys found: ['model_output', 'input', 'context'].Issues
Resolves an issue when it was not clear what's wrong if someone didn't add required key-value pair to task output dictionary because SDK was hiding this error.
Testing
Added unit tests
Documentation