search

dodona-edu / universal-judge

Universal judge for educational software testing
https://docs.dodona.be/en/tested
MIT License
9 stars 4 forks source link

Improve error message for YAML syntax errors #456

Closed niknetniko closed 7 months ago

niknetniko commented 7 months ago

For example, from https://dodona.be/en/submissions/15812470/.

If there is an error and we have the information, show a more "user friendly" error, including the actual line where the error occurred.

Before:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/niko/Ontwikkeling/universal-judge/tested/manual.py", line 56, in <module>
    run(config, sys.stdout)
  File "/home/niko/Ontwikkeling/universal-judge/tested/main.py", line 32, in run
    suite = parse_dsl(textual_suite)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niko/Ontwikkeling/universal-judge/tested/dsl/translate_parser.py", line 595, in parse_dsl
    dsl_object = _parse_yaml(dsl_string)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niko/Ontwikkeling/universal-judge/tested/dsl/translate_parser.py", line 117, in _parse_yaml
    return yaml.load(yaml_stream, loader)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/66wqmbkha5kg710zamqk0a9455vannnx-python3-3.11.4-env/lib/python3.11/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/66wqmbkha5kg710zamqk0a9455vannnx-python3-3.11.4-env/lib/python3.11/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "yaml/_yaml.pyx", line 673, in yaml._yaml.CParser.get_single_node
  File "yaml/_yaml.pyx", line 687, in yaml._yaml.CParser._compose_document
  File "yaml/_yaml.pyx", line 729, in yaml._yaml.CParser._compose_node
  File "yaml/_yaml.pyx", line 808, in yaml._yaml.CParser._compose_sequence_node
  File "yaml/_yaml.pyx", line 860, in yaml._yaml.CParser._parse_next_event
yaml.parser.ParserError: while parsing a block collection
  in "<unicode string>", line 1, column 1
did not find expected '-' indicator
  in "<unicode string>", line 2, column 1

Now:

YAML error while parsing test suite. This means there is a YAML syntax error.

The YAML parser indicates the problem lies at line 2, column 1:

    testcases:
    ^

The error message was:
    did not find expected '-' indicator while parsing a block collection

The detailed exception is provided below.
You might also find help by validating your YAML file with a YAML validator.

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/niko/Ontwikkeling/universal-judge/tested/manual.py", line 56, in <module>
    run(config, sys.stdout)
  File "/home/niko/Ontwikkeling/universal-judge/tested/main.py", line 32, in run
    suite = parse_dsl(textual_suite)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niko/Ontwikkeling/universal-judge/tested/dsl/translate_parser.py", line 595, in parse_dsl
    dsl_object = _parse_yaml(dsl_string)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niko/Ontwikkeling/universal-judge/tested/dsl/translate_parser.py", line 143, in _parse_yaml
    raise exc
  File "/home/niko/Ontwikkeling/universal-judge/tested/dsl/translate_parser.py", line 117, in _parse_yaml
    return yaml.load(yaml_stream, loader)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/66wqmbkha5kg710zamqk0a9455vannnx-python3-3.11.4-env/lib/python3.11/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/66wqmbkha5kg710zamqk0a9455vannnx-python3-3.11.4-env/lib/python3.11/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "yaml/_yaml.pyx", line 673, in yaml._yaml.CParser.get_single_node
  File "yaml/_yaml.pyx", line 687, in yaml._yaml.CParser._compose_document
  File "yaml/_yaml.pyx", line 729, in yaml._yaml.CParser._compose_node
  File "yaml/_yaml.pyx", line 808, in yaml._yaml.CParser._compose_sequence_node
  File "yaml/_yaml.pyx", line 860, in yaml._yaml.CParser._parse_next_event
yaml.parser.ParserError: while parsing a block collection
  in "<unicode string>", line 1, column 1
did not find expected '-' indicator
  in "<unicode string>", line 2, column 1