Closed KyleKing closed 2 years ago
this [curl, https://github.com/pycontribs/ruyaml/issues/new]
is perfectly valid yaml
You can check several parsers with the yaml-runtimes containers, in several ways:
echo '[curl, https://github.com/pycontribs/ruyaml/issues/new]' | docker run -i --rm yamlio/alpine-runtime-perl perl-refparser-event
Or you can use the playground https://play.yaml.io/main/parser?input=W2N1cmwsIGh0dHBzOi8vZ2l0aHViLmNvbS9weWNvbnRyaWJzL3J1eWFtbC9pc3N1ZXMvbmV3XQ==
A colon is only special if a whitespace is following (or if it is at the end of the line).
This was also fixed in pyyaml and libyaml a while ago:
edit: btw, the YAML terminology for list is sequence, and the [...]
syntax is called flow style
Thanks, I appreciate the extra background information!
Looks like my issue was with the pre-commit check-yaml
hook when run with "safe"
>>> import ruamel.yaml
>>>
>>> yaml_str = "- repos:\n - command: [curl, https://github.com/pycontribs/ruyaml/issues/new]\n"
>>>
>>> yaml = ruamel.yaml.YAML(typ='safe')
>>> yaml_dict = yaml.load(yaml_str)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/kyleking/Developer/wip/calcipy/.venv/lib/python3.10/site-packages/ruamel/yaml/main.py", line 434, in load
return constructor.get_single_data()
File "/Users/kyleking/Developer/wip/calcipy/.venv/lib/python3.10/site-packages/ruamel/yaml/constructor.py", line 119, in get_single_data
node = self.composer.get_single_node()
File "_ruamel_yaml.pyx", line 706, in _ruamel_yaml.CParser.get_single_node
File "_ruamel_yaml.pyx", line 724, in _ruamel_yaml.CParser._compose_document
File "_ruamel_yaml.pyx", line 773, in _ruamel_yaml.CParser._compose_node
File "_ruamel_yaml.pyx", line 850, in _ruamel_yaml.CParser._compose_sequence_node
File "_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node
File "_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node
File "_ruamel_yaml.pyx", line 773, in _ruamel_yaml.CParser._compose_node
File "_ruamel_yaml.pyx", line 850, in _ruamel_yaml.CParser._compose_sequence_node
File "_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node
File "_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node
File "_ruamel_yaml.pyx", line 773, in _ruamel_yaml.CParser._compose_node
File "_ruamel_yaml.pyx", line 852, in _ruamel_yaml.CParser._compose_sequence_node
File "_ruamel_yaml.pyx", line 904, in _ruamel_yaml.CParser._parse_next_event
ruamel.yaml.scanner.ScannerError: while scanning a plain scalar
in "<unicode string>", line 2, column 21
found unexpected ':'
in "<unicode string>", line 2, column 26
>>>
>>> yaml = ruamel.yaml.YAML(typ='safe')
>>> yaml_dicts = [*yaml.parse(yaml_str)]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/kyleking/Developer/wip/calcipy/.venv/lib/python3.10/site-packages/ruamel/yaml/main.py", line 348, in parse
while parser.check_event():
File "_ruamel_yaml.pyx", line 675, in _ruamel_yaml.CParser.check_event
File "_ruamel_yaml.pyx", line 529, in _ruamel_yaml.CParser._parse
ruamel.yaml.scanner.ScannerError: while scanning a plain scalar
in "<unicode string>", line 2, column 21
found unexpected ':'
in "<unicode string>", line 2, column 26
>>>
>>>
>>> # No error when not set to "safe"
>>> yaml = ruamel.yaml.YAML()
>>> yaml_dict = yaml.load(yaml_str)
>>> print(yaml_dict)
[ordereddict([('repos', [ordereddict([('command', ['curl', 'https://github.com/pycontribs/ruyaml/issues/new'])])])])]
>>>
Just so future readers are aware, check-yaml
hook currently uses ruamel
instead of ruyaml, and there's a ticket related to this issue at https://sourceforge.net/p/ruamel-yaml/tickets/248/, where they give a workaround of using the slower Python interpreter, for an issue apparently related to libyaml.
The workaround to fix the hook itself can be to downgrade the hook to v2.1.0, when it was still using pyYAML, using args: [--unsafe]
depending on your use case.
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.1.0
hooks:
- id: check-yaml
args: [--unsafe]
I initially reported this issue here: https://github.com/lyz-code/yamlfix/issues/161
But the root cause appears to be that
ruyaml
doesn't handle the edge case that certain characters (colons and possibly others?) need to be escaped within a list. A reproducible example is below:I'm not very familiar with the YAML spec and the
ruyaml
codebase, but I could take a look at contributing if someone could point me in the right direction