search

pratapbhanu / CRCNN

Classifying Relations by Ranking with Convolutional Neural Networks
58 stars 18 forks source link

what if the entity is multi-word instead of single word? #5

Closed eveliao closed 5 years ago

eveliao commented 5 years ago

I noticed that in dataio.py: i1 = newseq_words.index('entity_1_end') - 1#TODO: Use head of entity-phrase instead of rightmost word

why minus just 1? It seems to have conflict with the comment.

The original paper didn't mention the multi-word issue. But there are multi-word entities in the dataset(very few though). I wonder if the proposed model is better at single word entity than multi-word one.

Btw, the evaluation is:

            precision    recall  f1-score   support 
micro avg       0.50      0.50      0.50      2717
 macro avg       0.43      0.42      0.38      2717
weighted avg       0.47      0.50      0.45      2717

It's not quite good compared the results from paper. You said you used it in SemEval 2018, how was the performance? Can you provide some advice to improve it?

Thanks.

pratapbhanu commented 5 years ago

Hi Eveliao

That's correct, I used in SemEval2018, might have training logs somewhere, will get back to you if I find those. These might help you with exact training strategy/hyper-params. For the performance evaluation on SemEval2018 Task 7 see: Talla at SemEval-2018 Task 7

Regarding, i1 = newseq_words.index('entity_1_end') It is just taking the last(right-most) word of a multi-word entity. While in original paper, authors used the head-word of a multi-word entity. There is also one crucial difference from the original paper, this implementation does not implement Special Treatment of Artificial Classes (as described in the original paper).

Thanks!

ghezalahmad commented 5 years ago

Hi, I tried to run your model, I get this error, OSError: [E050] Can't find model 'en'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory. Do we have a directory with the name of "models". As it was not on your repository.

pratapbhanu commented 5 years ago

Hi It looks like Spacy error. Check this out and make sure you download spacy's model: https://github.com/explosion/spaCy/issues/1721

pratapbhanu commented 5 years ago

Hi @eveliao Sorry, I couldn't find the original training logs. I hope you got it working. Thanks!