search

attardi / deepnl

Deep Learning for Natural Language Processing
GNU General Public License v3.0
457 stars 116 forks source link

DeepNL NER issue I-ORG appears with out B-ORG #13

Closed kiran-surya closed 8 years ago

kiran-surya commented 8 years ago

Hi,

for the following sentence -

"Try One Maine, Two Maine When Williams flung the ball in to Stephenson, Cutler whistled the play dead and ruled five seconds, delighting the Knicks and their fans, shocking the Hornets and infuriating their coach, Steve Clifford."

Output: sentence=[(u'Try', u'O'), (u'One', u'O'), (u'Maine,', u'I-PER'), (u'Two', u'I-ORG'), (u'Maine', u'I-ORG'), (u'When', u'O'), (u'Williams', u'B-PER'), (u'flung', u'O'), (u'the', u'O'), (u'ball', u'O'), (u'in', u'O'), (u'to', u'O'), (u'Stephenson,', u'B-PER'), (u'Cutler', u'I-PER'), (u'whistled', u'O'), (u'the', u'O'), (u'play', u'O'), (u'dead', u'O'), (u'and', u'O'), (u'ruled', u'O'), (u'five', u'O'), (u'seconds,', u'O'), (u'delighting', u'O'), (u'the', u'O'), (u'Knicks', u'B-ORG'), (u'and', u'O'), (u'their', u'O'), (u'fans,', u'O'), (u'shocking', u'O'), (u'the', u'O'), (u'Hornets', u'B-ORG'), (u'and', u'O'), (u'infuriating', u'O'), (u'their', u'O'), (u'coach,', u'O'), (u'Steve', u'B-PER'), (u'Clifford.', u'I-PER')]

Here Two and Maine words are tagged as I-ORG. However there is no B-ORG prior to this in tagged sequence. Is this expected ???

attardi commented 8 years ago

It is unlikely to happen, since the tagger uses a Viterbi algorithm to ensure that the sequence of tags is proper, but it may happen. It does not happen to me on a model trained with CoNLL 2003 data.