dodona-edu / universal-judge

Universal judge for educational software testing
https://docs.dodona.be/en/tested
MIT License
9 stars 4 forks source link

Improve handling of unknown return types #376

Closed niknetniko closed 1 year ago

niknetniko commented 1 year ago

Currently, if a non-supported type is returned (i.e. a custom class or something TESTed doesn't support) we will show useless results to students.

For example, in the example from #355, the student implemented a property as a function, meaning the "return value" for the property call is a Function object. Currently, we report {"type":"unknown"} with a message Received {"type":"unknown"}, which caused Could not find valid type for {"type":"unknown"}. for get_values.. The reported return value and message are TESTed internals that should not be reported.

This is not very clear for students. In this PR, this is improved by attempting to show a useful string representation of the returned value (students can then compare this against the expected value), and if the string representation does not contain the type name (e.g. as with JavaScript), the type is also shown.

For the same example, the return value would be shown as [object Function], without any message. While the example is in JavaScript, this was implemented for all OOP languages we currently support.

Fixes #355.