search

chimpler / pyhocon

HOCON parser for Python
Apache License 2.0
493 stars 117 forks source link

Issue parsing HOCON with newline #324

Closed lannuttia closed 1 month ago

lannuttia commented 4 months ago

In 0.3.59, parsing the following would work fine.

from pyparsing import ConfigFactory

ConfigFactory.parse_string("a { c = 3\nd = 4 }")

As of 0.3.60, running that same script would result in the following exception being thrown.

pyparsing.exceptions.ParseException: Expected '}', found '='  (at char 12), (line:2, col:3)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/alannutt/Projects/lannuttia/dataconf/.venv/lib64/python3.8/site-packages/pyhocon/config_parser.py", line 192, in parse_string
    return ConfigParser().parse(content, basedir, resolve, unresolved_value)
  File "/home/alannutt/Projects/lannuttia/dataconf/.venv/lib64/python3.8/site-packages/pyhocon/config_parser.py", line 455, in parse
    config = config_expr.parseString(content, parseAll=True)[0]
  File "/home/alannutt/Projects/lannuttia/dataconf/.venv/lib64/python3.8/site-packages/pyparsing/util.py", line 256, in _inner
    return fn(self, *args, **kwargs)
  File "/home/alannutt/Projects/lannuttia/dataconf/.venv/lib64/python3.8/site-packages/pyparsing/core.py", line 1200, in parse_string
    raise exc.with_traceback(None)
pyparsing.exceptions.ParseSyntaxException: Expected '}', found '='  (at char 12), (line:2, col:3)

Interestingly enough, if I add a space in between the \n and the 'd', It parses without error again. I suspect that since the 'd' comes immediately after the newline, it is erroneously being interpreted as a control character but I don't really know.

pierresouchay commented 1 month ago

The problem is likely due to implicit parsing of dates... and likely due to a change in pyparsing.

Here is an minimal example:

from pyhocon import ConfigFactory

val = ConfigFactory.parse_string("""
a {
  y_min: 3
  y_max: 42
}
""")

assert val
assert val["a"]
print(val)
assert "y_min" in val["a"].keys()
assert "y_max" in val["a"].keys()
assert val["a"]["y_min"] == 3
assert val["a"]["y_max"] == 42

pyhocon==0.3.59, pyparsing==2.4.7

Works as expected!

pip install pyparsing==2.4.7 pyhocon==0.3.59 >/dev/null 2>/dev/null && python3 test_hocon.py
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])

pyhocon==0.3.60, pyparsing==2.4.7

Works as expected!

pip install pyparsing==2.4.7 pyhocon==0.3.49 >/dev/null 2>/dev/null && python3 test_hocon.py
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])

with pyhocon version 0.3.30 and versions of pyparsing >= 3.0.0

for pyhocon in 0.3.60; do for pyparsing in 2.4.7 3.0.0 3.0.1 3.0.2 3.0.3 3.0.4 3.0.5 3.0.6 3.0.7 3.0.8 3.0.9 3.1.0 3.1.1 3.1.2; do pip install pyparsing==$pyparsing pyhocon==$pyhocon >/dev/null 2>/dev/null && python test_hocon.py 2>/dev/null && echo "  RESULT: OK for pyhocon=$pyhocon pyparsing=$pyparsing" || echo "ERROR RESULT FAILED for pyhocon=$pyhocon pyparsing=$pyparsing"; done; done
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=2.4.7
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=3.0.0
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=3.0.1
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=3.0.2
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=3.0.3
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=3.0.4
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=3.0.5
ConfigTree([('a', ConfigTree([('y_min', 3), ('y_max', 42)]))])
  RESULT: OK for pyhocon=0.3.60 pyparsing=3.0.6

But if we continue with pyparsing ≥ 3.0.7...

But, from pyparsing version=3.0.7 ... 3.1.2 (lastest as today 2024-05-21):

ConfigTree([('a', ConfigTree([('y_min', 'relativedelta(years=+3)_max: 42')]))])
ERROR RESULT FAILED for pyhocon=0.3.60 pyparsing=3.0.7
ConfigTree([('a', ConfigTree([('y_min', 'relativedelta(years=+3)_max: 42')]))])
ERROR RESULT FAILED for pyhocon=0.3.60 pyparsing=3.0.8
ConfigTree([('a', ConfigTree([('y_min', 'relativedelta(years=+3)_max: 42')]))])
ERROR RESULT FAILED for pyhocon=0.3.60 pyparsing=3.0.9
ConfigTree([('a', ConfigTree([('y_min', 'relativedelta(years=+3)_max: 42')]))])
ERROR RESULT FAILED for pyhocon=0.3.60 pyparsing=3.1.0
ConfigTree([('a', ConfigTree([('y_min', 'relativedelta(years=+3)_max: 42')]))])
ERROR RESULT FAILED for pyhocon=0.3.60 pyparsing=3.1.1
ConfigTree([('a', ConfigTree([('y_min', 'relativedelta(years=+3)_max: 42')]))])
ERROR RESULT FAILED for pyhocon=0.3.60 pyparsing=3.1.2

It fails from version 3.0.6 of pyparsing. Likely, we should block those revisions!

pierresouchay commented 1 month ago

@lannuttia fix in https://github.com/chimpler/pyhocon/pull/326

pierresouchay commented 1 month ago

@lannuttia I think it has been fixed in 0.3.61

lannuttia commented 1 month ago

I can confirm that the issue that I was seeing was resolved after updating to v0.3.61. I appreciate the help here!

pierresouchay commented 1 month ago

@lannuttia Then, you might close the issue :-)

lannuttia commented 1 month ago

Alright, it's closed