KenKundert / nestedtext

Human readable and writable data interchange format
https://nestedtext.org
MIT License
362 stars 13 forks source link

Some weird but valid input is converted to invalid NestedText -- value becomes part of key #47

Closed AndydeCleyre closed 4 months ago

AndydeCleyre commented 4 months ago

I encountered this in the following roundabout way:

trouble.xml:

<?xml version="1.0" encoding="UTF-8"?>
<kcfg xmlns="http://www.kde.org/standards/kcfg/1.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.kde.org/standards/kcfg/1.0
                          http://www.kde.org/standards/kcfg/1.0/kcfg.xsd" >
    <kcfgfile name=""/>
    <group name="">
        <entry name="Duration" type="UInt">
            <default>250</default>
        </entry>
        <entry name="ExcludedWindowClasses" type="String">
            <default>krunner,yakuake</default>
        </entry>
    </group>
</kcfg>

I used xmljson to turn it into trouble.json:

{
  "{http://www.kde.org/standards/kcfg/1.0}kcfgfile": null,
  "{http://www.kde.org/standards/kcfg/1.0}group": {
    "{http://www.kde.org/standards/kcfg/1.0}entry": [
      {
        "{http://www.kde.org/standards/kcfg/1.0}default": 250
      },
      {
        "{http://www.kde.org/standards/kcfg/1.0}default": "krunner,yakuake"
      }
    ]
  }
}

This doesn't look great to me but it does seem to be valid JSON.

Then in Python:

In [1]: from pathlib import Path
In [2]: from json import loads as j_loads
In [3]: data = j_loads(Path('./trouble.json').read_text())
In [4]: from pprint import pprint
In [5]: pprint(data)
{'{http://www.kde.org/standards/kcfg/1.0}group': {'{http://www.kde.org/standards/kcfg/1.0}entry': [{'{http://www.kde.org/standards/kcfg/1.0}default': 250},
                                                                                                   {'{http://www.kde.org/standards/kcfg/1.0}default': 'krunner,yakuake'}]},
 '{http://www.kde.org/standards/kcfg/1.0}kcfgfile': None}
In [6]: from nestedtext import dumps as nt_dumps, loads as nt_loads
In [7]: print(nt_dumps(data))
: {http://www.kde.org/standards/kcfg/1.0}kcfgfile
: {http://www.kde.org/standards/kcfg/1.0}group
    : {http://www.kde.org/standards/kcfg/1.0}entry
        -
            : {http://www.kde.org/standards/kcfg/1.0}default250
        -
            : {http://www.kde.org/standards/kcfg/1.0}default
                > krunner,yakuake
In [8]: nt_loads(nt_dumps(data))
Unexpected exception formatting exception. Falling back to standard exception
Traceback (most recent call last):
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-10-f65fd089d142>", line 1, in <module>
    nt_loads(dumps(data))
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 1204, in loads
    loader = NestedTextLoader(lines, top, source, on_dup, keymap, normalize_key)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 821, in __init__
    self.values, self.keymap = self._read_value(0, ())
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 901, in _read_value
    return self._read_dict(depth, keys)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 987, in _read_dict
    value, loc = self._read_value(depth_of_next, new_keys)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 901, in _read_value
    return self._read_dict(depth, keys)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 987, in _read_dict
    value, loc = self._read_value(depth_of_next, new_keys)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 899, in _read_value
    return self._read_list(depth, keys)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 931, in _read_list
    value, loc = self._read_value(depth_of_next, new_keys)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 901, in _read_value
    return self._read_dict(depth, keys)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 993, in _read_dict
    report("multiline key requires a value.", line, None, colno=depth)
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/nestedtext/nestedtext.py", line 276, in report
    raise NestedTextError(template=message, *args, **kwargs)
nestedtext.nestedtext.NestedTextError: 5: multiline key requires a value.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/andy/.local/share/venvs/765b9c89d8e269a433a53f3fcbacba89/venv/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 2166, in showtraceback
    stb = value._render_traceback_()
TypeError: 'NoneType' object is not callable

Looks like the value became part of the key:

: {http://www.kde.org/standards/kcfg/1.0}default250
KenKundert commented 4 months ago

Thanks for reporting this bug. I believe I have fixed the issue in the latest version available on github (3.7.dev3).