search

lukeparser / pybison

Use Bison directly from Python
https://pypi.org/project/pybison
GNU General Public License v2.0
26 stars 6 forks source link

I try to run example, but failed #19

Closed x007007007 closed 3 years ago

x007007007 commented 3 years ago 
['A_END', 'A_START', 'BOOL', 'COLON', 'COMMA', 'FLOAT', 'INTEGER', 'O_END', 'O_START', 'STRING']
generating bison file: /tmp/pybison/pybison_JSONParser/tmp.y
bison cmd: bison -d -v -t -o tokens.c --verbose /tmp/pybison/pybison_JSONParser/tmp.y
CMD Output: b''
renaming bison output files
tokens.c => /tmp/pybison/pybison_JSONParser/tokens.c
tokens.h => /tmp/pybison/pybison_JSONParser/tokens.h
flex cmd: flex --header-file=lex.yy.h /tmp/pybison/pybison_JSONParser/tmp.l
CMD Output: b''
lex.yy.c => /tmp/pybison/pybison_JSONParser/tmp.lex.c
lex.yy.h => /tmp/pybison/pybison_JSONParser/tmp.lex.h
cc -g -D__declspec(x) -I/Users/xxc/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/include/python3.8 -c /tmp/pybison/pybison_JSONParser/tokens.c -o tmp/pybison/pybison_JSONParser/tokens.o
/tmp/pybison/pybison_JSONParser/tmp.y:10:17: warning: unknown attribute 'dllexport' ignored [-Wunknown-attributes]
__attribute__ ((dllexport)) char *rules_hash = "c0158e69cd6557b35364de90003f581abe1fbaa3";
                ^
tokens.c:910:9: error: redefinition of 'yylval' with a different type: 'void *' vs 'YYSTYPE' (aka 'int')
YYSTYPE yylval;
        ^
tokens.h:111:16: note: previous declaration is here
extern YYSTYPE yylval;
               ^
tokens.c:1118:50: warning: incompatible pointer types passing 'YYSTYPE *' (aka 'int *') to parameter of type 'void *const *' [-Wincompatible-pointer-types]
      YY_SYMBOL_PRINT ("Next token is", yytoken, &yylval, &yylloc);
                                                 ^~~~~~~
tokens.c:753:25: note: expanded from macro 'YY_SYMBOL_PRINT'
                  Kind, Value, Location); \
                        ^~~~~
tokens.c:788:64: note: passing argument to parameter 'yyvaluep' here
                 yysymbol_kind_t yykind, YYSTYPE const * const yyvaluep, YYLTYPE const * const yylocationp)
                                                               ^
tokens.c:1141:41: warning: incompatible pointer types passing 'YYSTYPE *' (aka 'int *') to parameter of type 'void *const *' [-Wincompatible-pointer-types]
  YY_SYMBOL_PRINT ("Shifting", yytoken, &yylval, &yylloc);
                                        ^~~~~~~
tokens.c:753:25: note: expanded from macro 'YY_SYMBOL_PRINT'
                  Kind, Value, Location); \
                        ^~~~~
tokens.c:788:64: note: passing argument to parameter 'yyvaluep' here
                 yysymbol_kind_t yykind, YYSTYPE const * const yyvaluep, YYLTYPE const * const yylocationp)
                                                               ^
tokens.c:1144:12: warning: incompatible integer to pointer conversion assigning to 'void *' from 'YYSTYPE' (aka 'int') [-Wint-conversion]
  *++yyvsp = yylval;
           ^ ~~~~~~
tokens.c:1553:32: warning: incompatible pointer types passing 'YYSTYPE *' (aka 'int *') to parameter of type 'void **' [-Wincompatible-pointer-types]
                      yytoken, &yylval, &yylloc);
                               ^~~~~~~
tokens.c:892:46: note: passing argument to parameter 'yyvaluep' here
            yysymbol_kind_t yykind, YYSTYPE *yyvaluep, YYLTYPE *yylocationp)
                                             ^
tokens.c:1615:12: warning: incompatible integer to pointer conversion assigning to 'void *' from 'YYSTYPE' (aka 'int') [-Wint-conversion]
  *++yyvsp = yylval;
           ^ ~~~~~~
tokens.c:1666:28: warning: incompatible pointer types passing 'YYSTYPE *' (aka 'int *') to parameter of type 'void **' [-Wincompatible-pointer-types]
                  yytoken, &yylval, &yylloc);
                           ^~~~~~~
tokens.c:892:46: note: passing argument to parameter 'yyvaluep' here
            yysymbol_kind_t yykind, YYSTYPE *yyvaluep, YYLTYPE *yylocationp)
                                             ^
/tmp/pybison/pybison_JSONParser/tmp.y:301:17: warning: unknown attribute 'dllexport' ignored [-Wunknown-attributes]
__attribute__ ((dllexport)) void do_parse(void *parser1,
                ^
8 warnings and 1 error generated.
Traceback (most recent call last):
  File "/Users/xxc/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/distutils/unixccompiler.py", line 117, in _compile
    self.spawn(compiler_so + cc_args + [src, '-o', obj] +
  File "/Users/xxc/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/distutils/ccompiler.py", line 910, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/Users/xxc/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/Users/xxc/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/distutils/spawn.py", line 157, in _spawn_posix
    raise DistutilsExecError(
distutils.errors.DistutilsExecError: command 'cc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test1.py", line 175, in <module>
    j = JSONParser(verbose=True, debugSymbols=True)
  File "test1.py", line 74, in __init__
    super(Parser, self).__init__(**kwargs)
  File "/Users/xxc/workspace/ply/venv/lib/python3.8/site-packages/bison/__init__.py", line 227, in __init__
    self.engine = ParserEngine(self)
  File "src/bison/cython/bison_.pyx", line 135, in bison.bison_.ParserEngine.__init__
  File "src/bison/cython/bison_.pyx", line 172, in bison.bison_.ParserEngine.openCurrentLib
  File "src/bison/cython/bison_.pyx", line 686, in bison.bison_.ParserEngine.buildLib
  File "/Users/xxc/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/Users/xxc/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'cc' failed with exit status 1

I use flex 2.6.4 and bison (GNU Bison) 3.7.4.

x007007007 commented 3 years ago

run example on ubuntu 

Traceback (most recent call last):
  File "javaparser.py", line 2194, in <module>
    main(*(sys.argv[1:]))
  File "javaparser.py", line 2189, in main
    p = Parser(verbose=verbose, keepfiles=keepfiles)
  File "/home/xxc/workspace/sql/venv/lib/python3.8/site-packages/bison/__init__.py", line 227, in __init__
    self.engine = ParserEngine(self)
  File "src/bison/cython/bison_.pyx", line 135, in bison.bison_.ParserEngine.__init__
  File "src/bison/cython/bison_.pyx", line 172, in bison.bison_.ParserEngine.openCurrentLib
  File "src/bison/cython/bison_.pyx", line 676, in bison.bison_.ParserEngine.buildLib
  File "/home/xxc/.pyenv/versions/3.8.0/lib/python3.8/shutil.py", line 409, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/xxc/.pyenv/versions/3.8.0/lib/python3.8/shutil.py", line 259, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: 'lex.yy.h'
sbrodehl commented 3 years ago

Hi @x007007007,

thanks for your feedback!

I looked into the issues a little bit and it seems, that

  1. flex not always produces the needed header file
  2. the lexer script included a definition of YYLVAL, which was not needed

I fixed both issue in a new branch fix-renaming, feel free to test it! However, there exists another problem with the example you used. It seems to me that the JSONParser is not capable of parsing the example file. :sweat_smile:

Anyway, we will test the changes a bit and will deploy a new version afterwards.