dodona-edu / universal-judge

Universal judge for educational software testing
https://docs.dodona.be/en/tested
MIT License
9 stars 4 forks source link

Investigate pylint error #463

Closed niknetniko closed 7 months ago

niknetniko commented 7 months ago

Standard Error:

Pylint crashed with
Traceback (most recent call last):
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/languages/python/linter.py", line 46, in run_pylint
    lint.Run(args, reporter=JSONReporter(output=pylint_out), do_exit=False)
TypeError: Run.__init__() got an unexpected keyword argument 'do_exit'
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/__main__.py", line 45, in <module>
    run(configuration, out)
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/main.py", line 38, in run
    judge(pack)
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/judge/core.py", line 187, in judge
    result_status, currently_open_tab = _process_results(
                                        ^^^^^^^^^^^^^^^^^
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/judge/core.py", line 325, in _process_results
    continue_ = evaluate_context_results(
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/judge/evaluation.py", line 296, in evaluate_context_results
    missing_stdout = _evaluate_channel(
                     ^^^^^^^^^^^^^^^^^^
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/judge/evaluation.py", line 113, in _evaluate_channel
    evaluation_result = evaluator(output, actual if actual else "")
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/oracles/programmed.py", line 54, in evaluate
    result = evaluate_programmed(
             ^^^^^^^^^^^^^^^^^^^^
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/judge/programmed.py", line 45, in evaluate_programmed
    return _evaluate_python(bundle, evaluator, expected, actual)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/X7G8DMYVScTX-2wZKYyjFA/judge/tested/judge/programmed.py", line 221, in _evaluate_python
    exec(f"__tested_test__result = {literal_function_call}", global_env)
  File "<string>", line 1, in <module>
  File "<string>", line 23, in evaluate_value
IndexError: string index out of range

Standard Output:

{"command": "start-judgement"}
{"message": "Pylint gecrasht", "command": "append-message"}
{"message": {"description": "Run.__init__() got an unexpected keyword argument 'do_exit'", "format": "code", "permission": "staff"}, "command": "append-message"}
{"title": "Feedback", "command": "start-tab"}
{"command": "start-context"}

https://dodona.be/en/submissions/16041449/

niknetniko commented 7 months ago

It seems that the internal error is not related to the pylint crash, e.g. https://dodona.be/en/submissions/16041495/.

It seems we use a deprecated parameter, which was removed in PyLint 3.0. The Docker images were "recently" updated to use pylint 3.0 in https://github.com/dodona-edu/docker-images/pull/301