Error in simuvex setup while building angr-dev

[tmilovanov]

Following all instructions from "Development install" section

git clone https://github.com/angr/angr-dev
cd angr-dev
mkvirtualenv angr
./setup.sh

I got

  Running setup.py develop for simuvex
    Running command /home/timcess/.virtualenvs/angr/bin/python2 -c "import setuptools, tokenize;__file__='/home/timcess/utils/angr-dev/simuvex/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps
    running develop
    Building sim_unicorn
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/timcess/utils/angr-dev/simuvex/setup.py", line 76, in <module>
        'simuvex': ['lib/*']
      File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/home/timcess/utils/angr-dev/simuvex/setup.py", line 47, in run
        self.execute(_build_sim_unicorn, (), msg='Building sim_unicorn')
      File "/usr/lib/python2.7/distutils/cmd.py", line 349, in execute
        util.execute(func, args, msg, dry_run=self.dry_run)
      File "/usr/lib/python2.7/distutils/util.py", line 309, in execute
        func(*args)
      File "/home/timcess/utils/angr-dev/simuvex/setup.py", line 19, in _build_sim_unicorn
        import pyvex
      File "/home/timcess/utils/angr-dev/pyvex/pyvex/__init__.py", line 29, in <module>
        pvc = _find_c_lib()
      File "/home/timcess/utils/angr-dev/pyvex/pyvex/__init__.py", line 21, in _find_c_lib
        ffi.cdef(_ffi_str)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/cffi/api.py", line 105, in cdef
        self._cdef(csource, override=override, packed=packed)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/cffi/api.py", line 119, in _cdef
        self._parser.parse(csource, override=override, **options)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/cffi/cparser.py", line 299, in parse
        self._internal_parse(csource)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/cffi/cparser.py", line 304, in _internal_parse
        ast, macros, csource = self._parse(csource)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/cffi/cparser.py", line 260, in _parse
        ast = _get_parser().parse(csource)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/cffi/cparser.py", line 40, in _get_parser
        _parser_cache = pycparser.CParser()
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pycparser/c_parser.py", line 87, in __init__
        outputdir=taboutputdir)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pycparser/c_lexer.py", line 66, in build
        self.lexer = lex.lex(object=self, **kwargs)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pycparser/ply/lex.py", line 911, in lex
        lexobj.readtab(lextab, ldict)
      File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pycparser/ply/lex.py", line 233, in readtab
        titem.append((re.compile(pat, lextab._lexreflags | re.VERBOSE), _names_to_funcs(func_name, fdict)))
      File "/home/timcess/.virtualenvs/angr/lib/python2.7/re.py", line 194, in compile
        return _compile(pattern, flags)
      File "/home/timcess/.virtualenvs/angr/lib/python2.7/re.py", line 249, in _compile
        p = sre_compile.compile(pattern, flags)
      File "/home/timcess/.virtualenvs/angr/lib/python2.7/sre_compile.py", line 583, in compile
        "sorry, but this version only supports 100 named groups"
    AssertionError: sorry, but this version only supports 100 named groups
Cleaning up...
  Removing source in /tmp/pip-build-L9GIGl/distribute
Command "/home/timcess/.virtualenvs/angr/bin/python2 -c "import setuptools, tokenize;__file__='/home/timcess/utils/angr-dev/simuvex/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps" failed with error code 1 in /home/timcess/utils/angr-dev/simuvex/
Exception information:
Traceback (most recent call last):
  File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pip/commands/install.py", line 317, in run
    prefix=options.prefix_path,
  File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pip/req/req_set.py", line 742, in install
    **kwargs
  File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 825, in install
    install_options, global_options, prefix=prefix)
  File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 981, in install_editable
    show_stdout=False)
  File "/home/timcess/.virtualenvs/angr/local/lib/python2.7/site-packages/pip/utils/__init__.py", line 718, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command "/home/timcess/.virtualenvs/angr/bin/python2 -c "import setuptools, tokenize;__file__='/home/timcess/utils/angr-dev/simuvex/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps" failed with error code 1 in /home/timcess/utils/angr-dev/simuvex/

This error occured on two different machines. It's probably new one because a couple of days ago I builded angr-dev with no problem at all.

[rhelmot]

ubuntu 16.04, I can't reproduce this :(

It looks like something nasty is going on with the automated parsing of the libvex headers. Can you upload /home/timcess/utils/angr-dev/pyvex/pyvex/vex_ffi.py for me to look at?

[tmilovanov]

I tried to build on Linux Mint 18 64bit

vex_ffi.zip