CogComp / cogcomp-nlpy

CogComp's light-weight Python NLP annotators
http://nlp.cogcomp.org/
Other
116 stars 26 forks source link

A problem about getting 'verb_srl' view with LocalPipeline. #120

Closed luyaojie closed 2 years ago

luyaojie commented 5 years ago

Hi,

I want to get 'verb_srl' view with LocalPipeline. However, I am meeting with a problem about edu.illinois.cs.cogcomp.sl.

from ccg_nlpy import local_pipeline

text = "Hello,  how are you.\n\n\n I am doing fine."

doc = [
    ["Hello", ",", "how", "are", "you", "."],
    ['I', 'am', 'doing', 'fine', '.']
]

local_p = local_pipeline.LocalPipeline()

local_tokenized_doc = local_p.doc(doc, pretokenized=True)
srl = local_tokenized_doc.get_srl_verb
print(srl)

Some error log about this problem:

Important Error Log:
... SOME LOG ...
Load trained Models.....
java.io.InvalidClassException: edu.illinois.cs.cogcomp.sl.core.SLModel; local class incompatible: stream classdesc serialVersionUID = 1, local class serialVersionUID = -2449880137698216590
    at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1885)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
    at edu.illinois.cs.cogcomp.sl.core.SLModel.loadModel(SLModel.java:81)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.initialize(DepAnnotator.java:68)
    at edu.illinois.cs.cogcomp.annotation.Annotator.doInitialize(Annotator.java:126)
    at edu.illinois.cs.cogcomp.annotation.Annotator.<init>(Annotator.java:106)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.<init>(DepAnnotator.java:55)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.<init>(DepAnnotator.java:51)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.<init>(DepAnnotator.java:41)
    at edu.illinois.cs.cogcomp.pipeline.main.PipelineFactory.buildAnnotators(PipelineFactory.java:305)
    at edu.illinois.cs.cogcomp.pipeline.main.PipelineFactory.buildPipeline(PipelineFactory.java:183)
    at edu.illinois.cs.cogcomp.pipeline.main.PipelineFactory.buildPipelineWithAllViews(PipelineFactory.java:227)
... SOME LOG ...
ERROR:ccg_nlpy.local_pipeline:Failed to add view SRL_VERB
ERROR:ccg_nlpy.local_pipeline:JVM exception occurred: edu.illinois.cs.cogcomp.sl.util.WeightVector.dotProduct(Ledu/illinois/cs/cogcomp/sl/util/IFeatureVector;I)F
INFO:ccg_nlpy.core.text_annotation:The view is the collection of the following views: ['LEMMA', 'NER_CONLL', 'PARSE_STANFORD', 'POS', 'SENTENCE', 'SHALLOW_PARSE', 'TOKENS']
Traceback (most recent call last):
  File "debug.py", line 16, in <module>
    srl = local_tokenized_doc.get_srl_verb
  File "/home/yaojie/anaconda2/envs/py36/lib/python3.6/site-packages/ccg_nlpy/core/text_annotation.py", line 165, in get_srl_verb
    return self.get_view("SRL_VERB")
  File "/home/yaojie/anaconda2/envs/py36/lib/python3.6/site-packages/ccg_nlpy/core/text_annotation.py", line 264, in get_view
    if not isinstance(self.view_dictionary[view_name], list):
KeyError: 'SRL_VERB'
nitishgupta commented 5 years ago

@danyaljj should be able to help you.

danyaljj commented 5 years ago

@luyaojie the error is weird, especially that it comes from the dependency parse package (not SRL). Can you try to see if you can reproduce this error, without pre-tokenized text? This is what I tried and it worked for me:

    from ccg_nlpy import local_pipeline

    text = "Hello,  how are you.\n\n\n I am doing fine."

    local_p = local_pipeline.LocalPipeline()

    local_tokenized_doc = local_p.doc(text)
    srl = local_tokenized_doc.get_srl_verb
    print(srl)
luyaojie commented 5 years ago
from ccg_nlpy import local_pipeline

text = "Hello,  how are you.\n\n\n I am doing fine."

doc = [
    ["Hello", ",", "how", "are", "you", "."],
    ['I', 'am', 'doing', 'fine', '.']
]

local_p = local_pipeline.LocalPipeline()

local_raw_doc = local_p.doc(text)
local_tokenized_doc = local_p.doc(doc, pretokenized=True)

print('Local Untokenized Text')
print(local_raw_doc.get_tokens)
print('Local Tokenized Text')
print(local_tokenized_doc.get_tokens)

print('Local Untokenized SRL')
try:
    srl = local_raw_doc.get_srl_verb
    print(srl)
except:
    print("Failed: Local Pipeline with Text")

print('Local Tokenized SRL')
try:
    srl = local_tokenized_doc.get_srl_verb
    print(srl)
except:
    print("Failed: Local Pipeline with Tokens")

Same error on this error.

But I can run 'verb_srl' on another machine B.

[failed]Machine A:

[successful] Machine B:

flackbash commented 3 years ago

I have a similar issue. I created a minimal example that comes with a Dockerfile so it will be easily reproducible for you. I attached all the necessary files: ccg_nlpy_test_files.zip

Maybe also related to #126 .

Steps to reproduce

This should produce the following output:

root@9f145b910d1d:/home# python3 ccg_nlpy_test.py /data/articles.txt 
INFO:ccg_nlpy.pipeline_config:Using local pipeline
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Load trained Models.....
java.io.InvalidClassException: edu.illinois.cs.cogcomp.sl.core.SLModel; local class incompatible: stream classdesc serialVersionUID = 1, local class serialVersionUID = -2449880137698216590
    at java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:689)
    at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2012)
    at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1862)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2169)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:493)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:451)
    at edu.illinois.cs.cogcomp.sl.core.SLModel.loadModel(SLModel.java:81)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.initialize(DepAnnotator.java:68)
    at edu.illinois.cs.cogcomp.annotation.Annotator.doInitialize(Annotator.java:126)
    at edu.illinois.cs.cogcomp.annotation.Annotator.<init>(Annotator.java:106)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.<init>(DepAnnotator.java:55)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.<init>(DepAnnotator.java:51)
    at edu.illinois.cs.cogcomp.depparse.DepAnnotator.<init>(DepAnnotator.java:41)
    at edu.illinois.cs.cogcomp.pipeline.main.PipelineFactory.buildAnnotators(PipelineFactory.java:305)
    at edu.illinois.cs.cogcomp.pipeline.main.PipelineFactory.buildPipeline(PipelineFactory.java:183)
    at edu.illinois.cs.cogcomp.pipeline.main.PipelineFactory.buildPipelineWithAllViews(PipelineFactory.java:227)
Reading POS tagger model from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [0.5 sec].
Loading parser from serialized file edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz ... done [0.5 sec].
INFO:ccg_nlpy.local_pipeline:pipeline has been set up
Article 0: Jerusalem 7-15 (AFP) -  A high level Israeli army official has said today Saturday that Israel believes Iran is set to begin acquiring nuclear capability for military purposes from 2005 and will be in a position to equip missiles with nuclear warheads, capable of reaching Israel, within ten years.   The official, who wished to remain anonymous, told journalists: "Iran will start accumulating nuclear capability for military purposes from 2005 and will be in a position, within the space of 10 years, to produce nuclear warheads that can be fitted to missiles capable of striking Israel."   The official made these statements after Tehran announced today Saturday that it had "successfully" completed a new test on the Shahab 3 ground-ground missile with a range of 1300 kilometers.   The official further said: "Iran is not capable of developing a nuclear program without Moscow's help."   He said that according to information obtained by Israel, tests on the Shahab 3 ground-ground missile were in fact "successful and reached their maximum range of 1300 kilometers which allows them to reach strategic centers in Israel."   The official said that the design of the Shahab missile is based on a North Korean missile called Nodong, itself developed from the Soviet Scud missile.   The official also stressed that Iran regards the "destruction of Israel as legitimate," saying that "Tehran is embroiled in strategic terrorist activities targeting Israel, especially alongside Palestinian and Lebanese organizations."   The official further considered that "the Shahab missile will not come into operation for two years and that an improved range of 2000 kilometers would only be achieved after this period."   Israel's Deputy Defense Minister, Ephraim Sneh, said today Saturday that Israel would "know how to find a suitable response" to the Iranian Shahab 3 ground-ground missile with a 1300 kilometer range.   Sneh said on army radio that "Israel will prepare itself to find a suitable response to this threat," without giving any further details.   The Iranian army announced today Saturday that it had "successfully" completed a new test on Shahab 3 ground-ground missiles.   An official at the Iranian Ministry of Defense told Tehran Radio that the "Shahab 3 defense missile was tested a second time to ensure it conforms to international standards."   In July 1998, the first test on this missile was announced. It is capable of reaching the majority of countries in the region, including Israel.  

ccgdoc:  <ccg_nlpy.core.text_annotation.TextAnnotation object at 0x7f0768023e80>
Downloading the folder from datastore . . . 
        GroupId: readonly.org.cogcomp.gazetteers
        ArtifactId: 1.6/gazetteers.zip
augmentedGroupId: readonly.org.cogcomp.gazetteers
versionedFileName: 1.6/gazetteers.zip
zippedFileName: /root/.cogcomp-datastore/tmp/1.6/gazetteers.zip
java.net.SocketTimeoutException: connect timed out
    at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
    at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
    at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.base/java.net.Socket.connect(Socket.java:609)
    at com.squareup.okhttp.internal.Platform.connectSocket(Platform.java:101)
    at com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:137)
    at com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:108)
    at com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
    at com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
    at com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
    at com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:281)
    at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
    at com.squareup.okhttp.Call.getResponse(Call.java:286)
    at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
    at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
    at com.squareup.okhttp.Call.execute(Call.java:80)
    at io.minio.MinioClient.execute(MinioClient.java:826)
    at io.minio.MinioClient.executeHead(MinioClient.java:1018)
    at io.minio.MinioClient.statObject(MinioClient.java:1154)
    at io.minio.MinioClient.getObject(MinioClient.java:1343)
    at org.cogcomp.Datastore.getDirectory(Datastore.java:556)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.TreeGazetteers.init(TreeGazetteers.java:72)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.TreeGazetteers.<init>(TreeGazetteers.java:51)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.GazetteersFactory.init(GazetteersFactory.java:54)
    at edu.illinois.cs.cogcomp.ner.LbjTagger.Parameters.readAndLoadConfig(Parameters.java:274)
    at edu.illinois.cs.cogcomp.ner.LbjTagger.Parameters.readConfigAndLoadExternalData(Parameters.java:56)
    at edu.illinois.cs.cogcomp.ner.NERAnnotator.initialize(NERAnnotator.java:111)
    at edu.illinois.cs.cogcomp.annotation.Annotator.doInitialize(Annotator.java:126)
    at edu.illinois.cs.cogcomp.annotation.Annotator.lazyAddView(Annotator.java:201)
    at edu.illinois.cs.cogcomp.annotation.Annotator.getView(Annotator.java:167)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.processBySentence(SentencePipeline.java:112)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.addView(SentencePipeline.java:88)
java.io.FileNotFoundException: /root/.cogcomp-datastore/tmp/1.6/gazetteers.zip (No such file or directory)
    at java.base/java.io.FileInputStream.open0(Native Method)
    at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
    at org.cogcomp.ZipHelper.unZipIt(ZipHelper.java:71)
    at org.cogcomp.Datastore.getDirectory(Datastore.java:585)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.TreeGazetteers.init(TreeGazetteers.java:72)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.TreeGazetteers.<init>(TreeGazetteers.java:51)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.GazetteersFactory.init(GazetteersFactory.java:54)
    at edu.illinois.cs.cogcomp.ner.LbjTagger.Parameters.readAndLoadConfig(Parameters.java:274)
    at edu.illinois.cs.cogcomp.ner.LbjTagger.Parameters.readConfigAndLoadExternalData(Parameters.java:56)
    at edu.illinois.cs.cogcomp.ner.NERAnnotator.initialize(NERAnnotator.java:111)
    at edu.illinois.cs.cogcomp.annotation.Annotator.doInitialize(Annotator.java:126)
    at edu.illinois.cs.cogcomp.annotation.Annotator.lazyAddView(Annotator.java:201)
    at edu.illinois.cs.cogcomp.annotation.Annotator.getView(Annotator.java:167)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.processBySentence(SentencePipeline.java:112)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.addView(SentencePipeline.java:88)
zippedFileName: /root/.cogcomp-datastore/tmp/1.6/gazetteers.zip
path: /root/.cogcomp-datastore/readonly.org.cogcomp.gazetteers/1.6/gazetteers
artifactId: gazetteers
java.io.FileNotFoundException: /root/.cogcomp-datastore/readonly.org.cogcomp.gazetteers/1.6/gazetteers/gazetteers/gazetteers-list.txt (No such file or directory)
    at java.base/java.io.FileInputStream.open0(Native Method)
    at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.TreeGazetteers.init(TreeGazetteers.java:73)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.TreeGazetteers.<init>(TreeGazetteers.java:51)
    at edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.GazetteersFactory.init(GazetteersFactory.java:54)
    at edu.illinois.cs.cogcomp.ner.LbjTagger.Parameters.readAndLoadConfig(Parameters.java:274)
    at edu.illinois.cs.cogcomp.ner.LbjTagger.Parameters.readConfigAndLoadExternalData(Parameters.java:56)
    at edu.illinois.cs.cogcomp.ner.NERAnnotator.initialize(NERAnnotator.java:111)
    at edu.illinois.cs.cogcomp.annotation.Annotator.doInitialize(Annotator.java:126)
    at edu.illinois.cs.cogcomp.annotation.Annotator.lazyAddView(Annotator.java:201)
    at edu.illinois.cs.cogcomp.annotation.Annotator.getView(Annotator.java:167)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.processBySentence(SentencePipeline.java:112)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.addView(SentencePipeline.java:88)
Downloading the folder from datastore . . . 
        GroupId: readonly.edu.illinois.cs.cogcomp.ner
        ArtifactId: 4.0/ner-model-enron-conll-all-data.zip
augmentedGroupId: readonly.edu.illinois.cs.cogcomp.ner
versionedFileName: 4.0/ner-model-enron-conll-all-data.zip
zippedFileName: /root/.cogcomp-datastore/tmp/4.0/ner-model-enron-conll-all-data.zip
java.net.SocketTimeoutException: connect timed out
    at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
    at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
    at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.base/java.net.Socket.connect(Socket.java:609)
    at com.squareup.okhttp.internal.Platform.connectSocket(Platform.java:101)
    at com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:137)
    at com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:108)
    at com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
    at com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
    at com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
    at com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:281)
    at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
    at com.squareup.okhttp.Call.getResponse(Call.java:286)
    at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
    at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
    at com.squareup.okhttp.Call.execute(Call.java:80)
    at io.minio.MinioClient.execute(MinioClient.java:826)
    at io.minio.MinioClient.executeHead(MinioClient.java:1018)
    at io.minio.MinioClient.statObject(MinioClient.java:1154)
    at io.minio.MinioClient.getObject(MinioClient.java:1343)
    at org.cogcomp.Datastore.getDirectory(Datastore.java:556)
    at edu.illinois.cs.cogcomp.ner.ModelLoader.load(ModelLoader.java:103)
    at edu.illinois.cs.cogcomp.ner.NERAnnotator.initialize(NERAnnotator.java:115)
    at edu.illinois.cs.cogcomp.annotation.Annotator.doInitialize(Annotator.java:126)
    at edu.illinois.cs.cogcomp.annotation.Annotator.lazyAddView(Annotator.java:201)
    at edu.illinois.cs.cogcomp.annotation.Annotator.getView(Annotator.java:167)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.processBySentence(SentencePipeline.java:112)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.addView(SentencePipeline.java:88)
java.io.FileNotFoundException: /root/.cogcomp-datastore/tmp/4.0/ner-model-enron-conll-all-data.zip (No such file or directory)
    at java.base/java.io.FileInputStream.open0(Native Method)
    at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
    at org.cogcomp.ZipHelper.unZipIt(ZipHelper.java:71)
    at org.cogcomp.Datastore.getDirectory(Datastore.java:585)
    at edu.illinois.cs.cogcomp.ner.ModelLoader.load(ModelLoader.java:103)
    at edu.illinois.cs.cogcomp.ner.NERAnnotator.initialize(NERAnnotator.java:115)
    at edu.illinois.cs.cogcomp.annotation.Annotator.doInitialize(Annotator.java:126)
    at edu.illinois.cs.cogcomp.annotation.Annotator.lazyAddView(Annotator.java:201)
    at edu.illinois.cs.cogcomp.annotation.Annotator.getView(Annotator.java:167)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.processBySentence(SentencePipeline.java:112)
    at edu.illinois.cs.cogcomp.pipeline.main.SentencePipeline.addView(SentencePipeline.java:88)
zippedFileName: /root/.cogcomp-datastore/tmp/4.0/ner-model-enron-conll-all-data.zip
path: /root/.cogcomp-datastore/readonly.edu.illinois.cs.cogcomp.ner/4.0/ner-model-enron-conll-all-data
artifactId: ner-model-enron-conll-all-data
ERROR:ccg_nlpy.local_pipeline:Failed to add view NER_CONLL
ERROR:ccg_nlpy.local_pipeline:JVM exception occurred: View NER_CONLL not found java.lang.IllegalArgumentException
INFO:ccg_nlpy.core.text_annotation:The view is the collection of the following views: ['SENTENCE', 'TOKENS']
Traceback (most recent call last):
  File "ccg_nlpy_test.py", line 15, in <module>
    ner_cons_list = ccgdoc.get_ner_conll.cons_list
  File "/usr/local/lib/python3.6/dist-packages/ccg_nlpy/core/text_annotation.py", line 138, in get_ner_conll
    return self.get_view("NER_CONLL")
  File "/usr/local/lib/python3.6/dist-packages/ccg_nlpy/core/text_annotation.py", line 264, in get_view
    if not isinstance(self.view_dictionary[view_name], list):
KeyError: 'NER_CONLL'

Any help on this is much appreciated.