Fatal Exception

[PatrickCrenshaw]

I'm getting an error when I try to run corenlp.py

$ python  corenlp/corenlp.py 
java -Xmx3g -cp stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0.jar:stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0-models.jar:stanford-corenlp-full-2013-11-12/xom.jar:stanford-corenlp-full-2013-11-12/joda-time.jar:stanford-corenlp-full-2013-11-12/jollyday.jar:stanford-corenlp-full-2013-11-12/ejml-0.23.jar edu.stanford.nlp.pipeline.StanfordCoreNLP -props /home/patrick/nlp-features/corenlp-python/corenlp/default.properties
Traceback (most recent call last):                                                                                
  File "corenlp/corenlp.py", line 575, in <module>
    nlp = StanfordCoreNLP(options.corenlp, properties=options.properties, serving=True)
  File "corenlp/corenlp.py", line 430, in __init__
    self._spawn_corenlp()
  File "corenlp/corenlp.py", line 413, in _spawn_corenlp
    self.corenlp.expect("done.", timeout=timeouts[i])  # Load model
  File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1316, in expect
    return self.expect_list(compiled_pattern_list, timeout, searchwindowsize)
  File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1330, in expect_list
    return self.expect_loop(searcher_re(pattern_list), timeout, searchwindowsize)
  File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1401, in expect_loop
    raise EOF (str(e) + '\n' + str(self))
pexpect.EOF: End Of File (EOF) in read_nonblocking(). Exception style platform.
<pexpect.spawn object at 0x8c50fac>
version: 2.4 ($Revision: 516 $)
command: /usr/bin/java
args: ['/usr/bin/java', '-Xmx3g', '-cp', 'stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0.jar:stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0-models.jar:stanford-corenlp-full-2013-11-12/xom.jar:stanford-corenlp-full-2013-11-12/joda-time.jar:stanford-corenlp-full-2013-11-12/jollyday.jar:stanford-corenlp-full-2013-11-12/ejml-0.23.jar', 'edu.stanford.nlp.pipeline.StanfordCoreNLP', '-props', '/home/patrick/nlp-features/corenlp-python/corenlp/default.properties']
searcher: searcher_re:
    0: re.compile("done.")
buffer (last 100 chars): 
before (last 100 chars): ld not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

after: <class 'pexpect.EOF'>
match: None
match_index: None
exitstatus: 1
flag_eof: True
pid: 18679
child_fd: 4
closed: False
timeout: 60
delimiter: <class 'pexpect.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 8192
ignorecase: False
searchwindowsize: 80
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1

[abendebury]

What is the output of pip freeze? How did you install the prerequisites for this package?

[PatrickCrenshaw]

I just did what the readme said to do (e.g. sudo pip install pexpect unidecode jsonrpclib ) with the modification of installing stanford-corenlp-full-2013-11-12 instead of stanford-corenlp-full-2013-06-20 because stanford-corenlp-full-2013-06-20 has different versions than what is expected. I tried both versions of stanford corenlp and they both gave this same error.

$ pip freeze BeautifulSoup==3.2.1 GnuPGInterface==0.3.2 Mako==0.7.1 MarkupSafe==0.15 MySQL-python==1.2.3 PAM==0.4.2 PIL==1.1.7 PyICU==1.4 Pygments==1.5 Theano==0.6.0rc3 Twisted-Core==12.2.0 Twisted-Names==12.2.0 Twisted-Web==12.2.0 UniConvertor==1.1.4 Unidecode==0.04.16 apt-xapian-index==0.44 aptoncd==0.1.98-bzr117-1.2 apturl==0.4.1ubuntu4 argparse==1.2.1 chardet==2.0.1 configglue==1.0 configobj==4.7.2 debtagshw==0.1 decorator==3.3.3 defer==1.0.6 dirspec==4.0.0 distribute==0.7.3 duplicity==0.6.19 feedparser==5.1.2 gufw==12.10.0 httplib2==0.7.4 ipython==0.13.1.rc2 joblib==0.6.4 jsonrpclib==0.1.3 lxml==2.3.5 matplotlib==1.1.1 nltk==3.0.0b1 nose==1.1.2 numexpr==2.0.1 numpy==1.7.1 oauth==1.0.1 openpyxl==1.5.8 pandas==0.8.0 paramiko==1.7.7.1 pexpect==2.4 piston-mini-client==0.7.2 protobuf==2.4.1 pyOpenSSL==0.13 pycrypto==2.6 pycups==1.9.62 pycurl==7.19.0 pyinotify==0.9.3 pyparsing==1.5.6 pyserial==2.5 pysmbc==1.0.13 python-apt==0.8.7ubuntu4.1 python-dateutil==1.5 python-debian==0.1.21-nmu2ubuntu1 pytz==2011k pyxattr==0.5.1 pyxdg==0.20 pyzmq==2.2.0 reportlab==2.5 scikit-learn==0.15.0b1 scipy==0.10.1 sessioninstaller==0.0.0 simplegeneric==0.8.1 simplejson==2.6.1 six==1.1.0 statsmodels==0.4.2 system-service==0.1.6 tables==2.3.1 tornado==2.3 uTidylib==0.2 versiontools==1.9.1 wsgiref==0.1.2 xlrd==0.6.1 xlwt==0.7.4 zope.interface==3.6.1

[abendebury]

Okay, well one thing you can do is to try to install the requirements in a virtualenv to make sure that the other python packages you have aren't conflicting.

sudo pip install virtualenv
virtualenv venv
source venv/bin/activate
pip install pexpect unidecode jsonrpclib

Then try running it again.

That said, are you sure you want to run this as a server? I'm not even sure if that functionality is bug free enough to be reliable. At least, in the wordseer project we aren't running it as a server.

[PatrickCrenshaw]

I actually get the same error when I run things like:

from corenlp import StanfordCoreNLP corenlp_dir = "stanford-corenlp-full-2013-11-12" corenlp = StanfordCoreNLP(corenlp_dir) # wait a few minutes... corenlp.raw_parse("He says that Joe killed Sam.")

I'll try the vitualenv.

[abendebury]

Yeah, raw_parse should be working. Let me know how the virtualenv works out.

[PatrickCrenshaw]

I've never used virtualenv before, so I may be doing something wrong, but it failed the same way when I did (output omitted for clarity):

sudo pip install virtualenv virtualenv venv source venv/bin/activate sudo venv/bin/pip install pexpect unidecode jsonrpclib venv/bin/python test.py

[abendebury]

Try to just do this:

rm -r virtualenv
virtualenv venv
source venv/bin/activate
pip install pexpect unidecode jsonrpclib
python test.py

Notice the lack of sudo when running pip install. Virtualenv should point pip and python to the appropriate executables.

[PatrickCrenshaw]

Didn't fix it, I'm afraid.

[abendebury]

You're getting the same error as in your first post?

What about the raw_parse method?

[PatrickCrenshaw]

Sorry I wasn't clear.

Yes, I am getting the same error whether I try to run a server or I use the raw_parse method, which I am calling in a file I named "test.py".

[abendebury]

Can you paste the contents of test.py?

[PatrickCrenshaw]

from corenlp import StanfordCoreNLP corenlp_dir = "stanford-corenlp-full-2013-11-12" corenlp = StanfordCoreNLP(corenlp_dir) # wait a few minutes... corenlp.raw_parse("He says that Joe killed Sam.")

[abendebury]

Do you have java installed? Which version?

[PatrickCrenshaw]

$ java -version java version "1.7.0_51" OpenJDK Runtime Environment (IcedTea 2.4.4) (7u51-2.4.4-0ubuntu0.12.10.2) OpenJDK Server VM (build 24.45-b08, mixed mode)

I'll try sun java and see if that makes a difference.

[PatrickCrenshaw]

Changed java version and I still get the same error.

New java version: $ java -version java version "1.8.0_20" Java(TM) SE Runtime Environment (build 1.8.0_20-b26) Java HotSpot(TM) Server VM (build 25.20-b23, mixed mode)

[abendebury]

Does the error happen if you type the commands into the python console?

[PatrickCrenshaw]

yep.

$ python
Python 2.7.3 (default, Feb 27 2014, 19:39:10) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from corenlp import StanfordCoreNLP
>>> corenlp_dir = "stanford-corenlp-full-2013-11-12"
>>> corenlp = StanfordCoreNLP(corenlp_dir)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "corenlp/corenlp.py", line 430, in __init__
    self._spawn_corenlp()
  File "corenlp/corenlp.py", line 419, in _spawn_corenlp
    self.corenlp.expect("\nNLP> ")
  File "/home/patrick/nlp-features/corenlp-python/venv/local/lib/python2.7/site-packages/pexpect/__init__.py", line 1451, in expect
    timeout, searchwindowsize)
  File "/home/patrick/nlp-features/corenlp-python/venv/local/lib/python2.7/site-packages/pexpect/__init__.py", line 1466, in expect_list
    timeout, searchwindowsize)
  File "/home/patrick/nlp-features/corenlp-python/venv/local/lib/python2.7/site-packages/pexpect/__init__.py", line 1554, in expect_loop
    raise EOF(str(err) + '\n' + str(self))

...and so on...

[abendebury]

Perhaps you aren't using the correct version of the java package? Try this file. Make sure you delete the old java files and that corenlp_dir is a correct path to the directory.

[ghost]

Replicated this error with java 1.8.0_45, pip freeze

Unidecode==0.04.18
argparse==1.2.1
jsonrpclib==0.1.3
pexpect==3.3
wsgiref==0.1.2

and directory structure ~/.../stanford-corenlp-python/{corenlp, stanford-corenlp-full-2015-04-20, ...}

any developments since last year?

[daimrod]

Hi,

Just to say that I had the exact same error using stanford-corenlp-full-2015-04-20 until I switch to java 1.8.

Thanks for your work!

[richardxiong]

Hello, I encounter the same error using stanford-corenlp-full-2015-04-20. And I'm using java 1.8.0_77: $ java -version java version "1.8.0_77" Java(TM) SE Runtime Environment (build 1.8.0_77-b03)

Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 12:54:16) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information.

corenlp_dir = "stanford-corenlp-full-2015-04-20" from corenlp import StanfordCoreNLP corenlp = StanfordCoreNLP(corenlp_dir) Traceback (most recent call last): File "", line 1, in File "corenlp/corenlp.py", line 346, in init self._spawn_corenlp() File "corenlp/corenlp.py", line 335, in _spawn_corenlp self.corenlp.expect("\nNLP> ") File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pexpect/spawnbase.py", line 321, in expect timeout, searchwindowsize, async) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pexpect/spawnbase.py", line 345, in expect_list return exp.expect_loop(timeout) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pexpect/expect.py", line 105, in expect_loop return self.eof(e) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pexpect/expect.py", line 50, in eof raise EOF(msg) pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform. <pexpect.pty_spawn.spawn object at 0x1028ed150> command: /usr/bin/java args: ['/usr/bin/java', '-Xmx3g', '-cp', 'edu.stanford.nlp.pipeline.StanfordCoreNLP', '-props', '/Users/richard_xiong/Documents/DeepLearningMaster/stanford-corenlp-python-master/corenlp/default.properties'] buffer (last 100 chars): '' before (last 100 chars): 'ld not create the Java Virtual Machine.\r\nError: A fatal exception has occurred. Program will exit.\r\n' after: <class 'pexpect.exceptions.EOF'> match: None match_index: None exitstatus: None flag_eof: True pid: 9849 child_fd: 5 closed: False timeout: 30 delimiter: <class 'pexpect.exceptions.EOF'> logfile: None logfile_read: None logfile_send: None maxread: 8192 ignorecase: False searchwindowsize: 80 delaybeforesend: 0.05 delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_re: 0: re.compile(" NLP> ")

Could you help me on this?

[muety]

Any news about this? I'm getting the same error.

[vuduc153]

Just gonna leave this here for future reference. Got the same issue running the package on Google Colab. I managed to fix the issue using:

!apt-get install openjdk-8-jdk-headless -qq > /dev/null
import os
os.environ["JAVA_HOME"] = "/usr/lib/jvm/java-8-openjdk-amd64"
!update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java