search

dair-iitd / OpenIE-standalone

Other
565 stars 73 forks source link

EOF exception while loading languageModel #18

Open mssharma5523 opened 6 years ago

mssharma5523 commented 6 years ago

Hi, While running the program on Windows 10, I get EOF exception when the language model is loaded in the ListExtractorLanguageModelBasedExtractor class. This exception is thrown by the berkeley.nlp.lm.StupidBackoffLm package being used in the class. Can this be looked at? Is there any other alternative to avoid this error?
image

Regards

swarnaHub commented 6 years ago

Please note Open IE 5 hasn't been tested on a windows machine. You might be getting this error because I had compiled the Language Model is an ubuntu system. I can take a look at it, if given some time. For the time being, I'd suggest you to try Open IE on ubuntu or mac OS.

mssharma5523 commented 6 years ago

Hi @swarnaHub , Thanks for the response. I am primarily a windows user and would love if this can work in Windows machine also. After lots of debugging from my side, I am sure the problem is with the languageModel file deserialization. Can you please generate a languageModel compiled on windows machine. Else, can you tell me the steps on how I can generate the languageModel locally(if possible)

swarnaHub commented 6 years ago

I have compiled the Language Model into a Stupid Backoff Object and have uploaded the serialized file. You can create it the following way:

  1. Download the Web1T file and the vocabulary for English from http://tomato.banatao.berkeley.edu:8080/berkeleylm_binaries/

  2. Download the Language Model jar from https://code.google.com/archive/p/berkeleylm/downloads.

  3. Read the files using the method readGoogleLmBinary into a Stupid Backoff Object. The source of the method is here - https://github.com/adampauls/berkeleylm/blob/master/src/edu/berkeley/nlp/lm/io/LmReaders.java

  4. Serialize that object and you will get the 6.4G file that I uploaded.

shubhamnirbhay commented 5 years ago

Hi I am unable to download the languageModel file and unable to understand how to generate it locally. Can you please explain the procedure?