Let teams compete by making them create hard instances and fast solvers for problems of your choice. Then pitch these instances and solvers against one another. All language-agnostic.
If a solver crashes during a fight, the JSON file containing the match results displays a corresponding message for that fight. I was able to influence what exactly that message contains from within the program.
The Solver used for the demonstration is written in C++ and solves the Pairsum problem. At problem sizes greater than or equal to 100, instead of solving the instance, it throws an exception that escapes the main method, crashing the program. The message of this exception contains the instance that was given to it, formatted in JSON. Given below is a result from a match, where this modified Solver was used.
This demonstrates that it is possible to encode a problem instance (or any other sensible information) in match results. I suggest one of the following actions:
remove the message (or rather the detailed description)
make it not customizable
point this out to possible organizers using the framework, so that they can decide if this exploit should be allowed or not
If a solver crashes during a fight, the JSON file containing the match results displays a corresponding message for that fight. I was able to influence what exactly that message contains from within the program. The Solver used for the demonstration is written in C++ and solves the Pairsum problem. At problem sizes greater than or equal to 100, instead of solving the instance, it throws an exception that escapes the main method, crashing the program. The message of this exception contains the instance that was given to it, formatted in JSON. Given below is a result from a match, where this modified Solver was used. This demonstrates that it is possible to encode a problem instance (or any other sensible information) in match results. I suggest one of the following actions: