vlasovskikh / funcparserlib

Recursive descent parsing library for Python based on functional combinators
https://funcparserlib.pirx.ru
MIT License
338 stars 38 forks source link

`.parse` doesn't support arbitrary iterables #78

Open Kodiologist opened 1 year ago

Kodiologist commented 1 year ago
>>> from funcparserlib.parser import a, many
>>> many(a("x")).parse("xxxxxxx")
['x', 'x', 'x', 'x', 'x', 'x', 'x']
>>> many(a("x")).parse(x for x in "xxxxxxx")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hippo/Scratch/Python-venv/lib/python3.11/site-packages/funcparserlib/parser.py", line 221, in parse
    (tree, _) = self.run(tokens, State(0, 0, None))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hippo/Scratch/Python-venv/lib/python3.11/site-packages/funcparserlib/parser.py", line 569, in _many
    (v, s) = p.run(tokens, s)
             ^^^^^^^^^^^^^^^^
  File "/home/hippo/Scratch/Python-venv/lib/python3.11/site-packages/funcparserlib/parser.py", line 614, in _some
    if s.pos >= len(tokens):
                ^^^^^^^^^^^
TypeError: object of type 'generator' has no len()

It seems likely one would want to parse an iterable stream of tokens at times.