search

BLLIP / bllip-parser

BLLIP reranking parser (also known as Charniak-Johnson parser, Charniak parser, Brown reranking parser) See http://pypi.python.org/pypi/bllipparser/ for Python module.
http://bllip.cs.brown.edu/
227 stars 53 forks source link

Python 3.5 support #47

Open wbecker opened 8 years ago

wbecker commented 8 years ago

When I try to build it with Python 3.5 I get the following error

Collecting bllipparser
  Using cached bllipparser-2015.12.3.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python3.5/site-packages (from bllipparser)
Installing collected packages: bllipparser
  Running setup.py install for bllipparser ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-84szdo98/bllipparser/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-hkxweaja-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.5
    creating build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/CharniakParser.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/ModelFetcher.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/RerankingParser.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/ParsingShell.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/__init__.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/JohnsonReranker.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/Utility.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/RerankerFeatureCorpus.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/__main__.py -> build/lib.linux-x86_64-3.5/bllipparser
    running build_ext
    building 'bllipparser._CharniakParser' extension
    creating build/temp.linux-x86_64-3.5
    creating build/temp.linux-x86_64-3.5/first-stage
    creating build/temp.linux-x86_64-3.5/first-stage/PARSE
    creating build/temp.linux-x86_64-3.5/first-stage/PARSE/swig
    gcc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fPIC -Ifirst-stage/PARSE/ -I/usr/include/python3.5m -c first-stage/PARSE/swig/wrapper.C -o build/temp.linux-x86_64-3.5/first-stage/PARSE/swig/wrapper.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    first-stage/PARSE/swig/wrapper.C:30900:1: error: too many initializers for ‘PyAsyncMethods’
     };
     ^
    ... more errors ensue....

I've never touched swig in my life, but looking at this it's seems that wrapper.C is not being regenerated properly.

dmcc commented 8 years ago

Thanks for the report. swig had problems generating the wrapper code for Python 3.5 (https://github.com/BLLIP/bllip-parser/commit/d183ab8a2896371af9e1c3c15c3e2ea0d486d4c7) but version 3.0.8 claims to fix these (haven't had time to test yet). Are you running the latest version? (swig -version)

(side note: I will update docs to make these issues clearer)

wbecker commented 8 years ago

I tried it with the latest version and it failed in just the same way.

I backed down and installed python 3.4 and swig 3.0.7 and it's all working now!

jarussell commented 8 years ago

I am having the same (or a similar) problem.

python --version
Python 3.5.1

swig -version|grep -v ^$
SWIG Version 3.0.8
Compiled with g++ [x86_64-unknown-linux-gnu]
Configured options: +pcre
Please see http://www.swig.org for reporting bugs and further information

no problems with the C++ code itself, seems to build fine with:

git clone https://github.com/BLLIP/bllip-parser
cd bllip-parser
make

This code builds but I am new to python and not sure how to get it to install from pip.

sudo pip install bllipparser --no-cache-dir #couldn't figure out how to force it to redownload since I am new to python

gives the same or similar errors to wbecker.

tox appears to build a 2.7 (successfully) and 3.4 version but is unable to build the 3.4 since I don't have 3.4 on my system.

dmcc commented 8 years ago

I think the issue is that the wrapper file bundled with the 2015.12.3 release uses SWIG 3.0.7 or earlier, which causes these build problems. I will make a new release soon using SWIG 3.0.8 to generate the wrapper file which will hopefully address this problem.

In the meantime, here's a possible workaround::

shell> make real-clean
shell> python setup.py build
shell> sudo python setup.py install

The first command will wipe out the SWIG-generated wrapper files (among other things) which will be regenerated during building.

halfak commented 8 years ago

I'm still running into this issue running python 3.5.1 on Ubuntu 16.04. pip pulls down bllipparser-2015.12.3.tar.gz.

dmcc commented 8 years ago

Thanks for the note @halfak. Sorry, this fell off my plate. I'll try to get a new-enough version of SWIG working this weekend and cut a new release.

dmcc commented 8 years ago

Alright, there's a new release (2016.9.11) which should work with Python 3.5/swig 3.0.8. Please let me know either way if it works or doesn't. Thanks!

anupamme commented 7 years ago

Hello @dmcc

I tried installing it and it fails:

ubuntu - 16.04 (xenial) python - 3.5.2 swig - 3.0.8

Can @halfak confirm whether you can install because you are able to install

halfak commented 7 years ago

@anupamme, I am able to install bllipparser-2016.9.11 via pypi with pip.

ubuntu - 16.04 python - 3.5.1 swig - 3.0.8

anupamme commented 7 years ago

Okay I am not able to install. These are the things I have tried:

  1. Using pypi, installing bllipparser-2016.9.11
  2. git cloned this repo and ran python setup.py build, but got same errors.

I have tried it on 2 machines:

  1. ubuntu - 16.04 (xenial) python - 3.5.2 swig - 3.0.8

  2. Mac OS 10.11.5 (el capitan) python - 3.5.0 swig - 3.0.10

So I am not sure what else I can try. So if someone can suggest that would be great.

dmcc commented 7 years ago

@anupamme Which errors are you seeing?

anupamme commented 7 years ago

On ubuntu 16.04 (installing using pypi):

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Ifirst-stage/PARSE/ -I/usr/include/python3.5m -c first-stage/PARSE/swig/wrapper.C -o build/temp.linux-x86_64-3.5/first-stage/PARSE/swig/wrapper.o cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ first-stage/PARSE/swig/wrapper.C:172:21: fatal error: Python.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3.5 -u -c "import setuptools, tokenize;file='/tmp/pip-build-vn848n2m/bllipparser/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-j_za0ero-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-vn848n2m/bllipparser/

On Mac 10.11.5 (installing using pypi):

/usr/bin/clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c second-stage/programs/features/simple-api.cc -o build/temp.macosx-10.6-intel-3.5/second-stage/programs/features/simple-api.o -iquote second-stage/programs/features/ -DSWIGFIX -std=c++11 In file included from second-stage/programs/features/simple-api.cc:24: In file included from second-stage/programs/features/sp-data.h:33: In file included from second-stage/programs/features/sptree.h:22: In file included from second-stage/programs/features/heads.h:36: In file included from second-stage/programs/features/tree.h:144: second-stage/programs/features/utility.h:56:10: fatal error: 'forward_list' file not found

include 

         ^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
----------------------------------------

Command "/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;file='/private/tmp/pip-build-m2ez9evu/bllipparser/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-ur28p0hx-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-m2ez9evu/bllipparser/

dmcc commented 7 years ago

For Ubuntu, looks like you're missing Python.h. Try using apt-get to install libpython3.5-dev.

http://stackoverflow.com/questions/16880571/why-does-clang-lack-forward-list might be part of what's going on with Mac OS 10.11.5, but it's not clear to me why this works for others.

anupamme commented 7 years ago

Works on Ubuntu after installing libpython3.5-dev.

On mac I am unsure why this is not working but may be I have beta version of xcode 8 and that is causing it.

But I can live with it working on ubuntu so I am good to go. Thanks for the prompt help.