Support for running tests against some code

[anandology]

One of the possible uses of Falcon would be to verify of the submitted code is correct by running tests against it.

One approach to make this possible is by enabling two different modes for each runtime exec and test. The exec would be the default option with the existing behavior. The test mode provides the output as JSON.

API

Exeucute come code:

$ curl https://falcon/runtimes/python -d 'print("helloworld")'
helloworld

Test some code:

$ curl -F main.py=@main.py -F test_main.py=@test_main.py https://falcon/runtimes/python 
{
    "testcases": [
        {
            "name": "test_square",
            "filename": "test_main.py",
            "description": "Test square with simple input",
            "time_taken": 0.001,
            "status": "success"
        },
        {
            "name": "test_square_ng",
            "filename": "test_main.py",
            "description": "Test square with negative numbers",
            "time_taken": 0.001,
            "status": "failure",
            "error_message": "NameError: name `doom` is not defined".
            "error_detail": "..."
        }
    ],
    "stats": {
        "tests": 2,
        "success": 1
        "failure": 1,
        "time_taken": 0.01
    },
    "status": "failure"
}

[anandology]

done. Implemented by adding multiple modes. The default mode is exec and the other mode is test.

The API is slightly different than proposed.

$ curl -H 'x-falcon-mode: test' -F square.py=@square.py -F test_square.py=@test_square.py http://localhost:8010/runtimes/python
{
 "testcases": [
  {
   "filename": "test_square.py",
   "name": "test_square",
   "time_taken": "0.001",
   "outcome": "passed"
  }
 ],
 "stats": {
  "tests": "1",
  "passed": 1,
  "failed": 0,
  "time_taken": "0.049"
 }
}