search

inducer / pycparserext

Extensions for Eli Bendersky's pycparser
http://pypi.python.org/pypi/pycparserext
Other
83 stars 28 forks source link

PLY3.6 compatibility #9

Closed teskje closed 9 years ago

teskje commented 9 years ago

An exception is thrown when trying to instantiate a GnuCParser:

>>> from pycparserext.ext_c_parser import GnuCParser
>>> p = GnuCParser()
ERROR: /home/j0ker/pycparserext/pycparserext/ext_c_lexer.py:11: No regular expression defined for rule 't_FLOAT_CONST'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/j0ker/pycparserext/pycparserext/ext_c_parser.py", line 32, in __init__
self.clex.build()
  File "/usr/lib/python3.4/site-packages/pycparser-2.12-py3.4.egg/pycparser/c_lexer.py", line 66, in build
  File "/usr/lib/python3.4/site-packages/pycparser-2.12-py3.4.egg/pycparser/ply/lex.py", line 894, in lex
SyntaxError: Can't build lexer

This also leads to 4 of 6 tests failing and makes the whole library unusable.

I am using Python 2.7.9 on an up-to-date Arch Linux. pycparserext and its dependencies were installed from a freshly cloned repo via setup.py.

inducer commented 9 years ago

Contrary to your statement, it looks like you're using Python 3.4. I can reproduce your issue using Python 3, but not using Python 2.7.

I've traced this back to an error that's being reported (only on Py3):

"ERROR: /home/andreas/src/pycparserext/pycparserext/ext_c_lexer.py:11: No regular expression defined for rule                                                                       
't_FLOAT_CONST'\n"

This is confusing to me, because there's an @TOKEN decorator specifying a regex right above t_FLOAT_CONST. That's all the time I've got for troubleshooting this right now. Hope this helps, and we'll leave this bug open to track Py3 compatibility.

teskje commented 9 years ago

Actually, the system I worked on has Python 3.4 as default, but as I figured just like you that this might be Python3-related, I verified it with Python 2.7 also. I just reproduced the issue on my laptop. To prove, that I am not crazy:

lexer-error

Also, the tests don't use the print function, so since I got failing tests, I must have used Python 2.7.

So I think, this is not a Python3 problem. My best guess would be that there recently was some API change in PLY that broke backwards-compatibility. Have you tried updating PLY to verify the bug?

inducer commented 9 years ago

The real problem is that pycparser ships with its own copy of ply. Fixed in revision f04898f.