search

dalab / end2end_neural_el

Apache License 2.0
231 stars 68 forks source link

How to combine the model with the Stanford NER system? #4

Closed Jorigorn closed 5 years ago

Jorigorn commented 6 years ago

Dear @octavian-ganea , Thanks for sharing the code,

I am trying the reproducing the results from the paper.

How to combine the model with the Stanford NER system?

Thanks. For the evalueation the pretrained models: Everything goes well, local evaluation and for printing the annotated datasets and Trying the system on random user input text without any problem.

but in Gerbil evaluation section, 3 terminasl started without errors, but after Run experiment The website shows, the experiment is running, after around 10 mins, refresh, it shows:

GerbilException: Saw to many errors (maximum was set to 15). (error type -108: The annotator caused too many single errors.)

2018-11-13 14:04:58,210 [pool-1-thread-1] ERROR [org.aksw.gerbil.annotator.decorator.ErrorCountingAnnotatorDecorator] - <Got an Exception from the annotator (13_11_v4_f ixedNLtk (NIF WS)): Exception while sending request.> 2018-11-13 14:04:58,211 [pool-1-thread-1] ERROR [org.aksw.gerbil.execute.ExperimentTask] - GerbilException: Saw to many errors (maximum was set to 15). (error type -108: The annotator caused too many single errors.) at org.aksw.gerbil.annotator.decorator.ErrorCountingAnnotatorDecorator.increaseErrorCount(ErrorCountingAnnotatorDecorator.java:561)

I am tring to incorate the transformer(like google Bert) to the system, see if there will be some improvement.

Thanks a lot. :) John

NikosKolitsas commented 5 years ago

Here are instructions on how to run the system for the different tasks. When you run the EL command then on Gerbil experiment configuration page you should choose "experiment type" A2KB. When you run the ED command then you have to choose D2KB, and when you run the third command (our system for ED coupled with the stanford ner) then again you have to choose experiment type A2KB. I repeat here the command for running the ED model with the Stanford NER: python -m gerbil.server --training_name=base_att_global --experiment_name=paper_models \ --persons_coreference_merge=True --ed_mode --entity_extension=extension_entities \ --el_with_stanfordner_and_our_ed=True

My guess for your error is that you run the above command but you selected wrong experiment type (the D2KB instead of A2KB) since ED + Stanford NER implement the EL task.