search

guillaumegenthial / tf_ner

Simple and Efficient Tensorflow implementations of NER models with tf.estimator and tf.data
Apache License 2.0
923 stars 275 forks source link

Prediction shows Inside before Begin or Without Begin for some predictions #17

Closed varshachawan closed 5 years ago

varshachawan commented 5 years ago

@guillaumegenthial training Dataset used was In B-I-O format Begin Inside Out format but for some of the predictions it shows I-LOC without B-LOC If I assume correctly, Inside should be always after Begin ... Any thoughts where it is going wrong? Is there any way to control this transitions

Same problem with sequence_tagging implementation and this one Any suggestions where to control these transitions ?

guillaumegenthial commented 5 years ago

Hi @varshachawan, I would say that nothing is wrong, it just turns out that because the I tag is more frequent, in some cases the optimal choice as seen by the model is to pick the I tag instead of the B tag, even though the CRF transition scores might penalize the absence of a B tag.

To solve this, one option would be to set the scores of unknown transitions of the CRF to some very low number so that unseen patterns will never get predicted.

This could be done once upon parameter initialization.

varshachawan commented 5 years ago

@guillaumegenthial ,

  1. How can I set the scores of unknown transitions. Following are tags --> B-LOC , I-LOC , B-PER, I-PER if nothing then tagged as O [Other] crf_log_likelihood accepts transition_params=None tf.contrib.crf.crf_log_likelihood( inputs, tag_indices, sequence_lengths, transition_params=None )

If I refere your sequence_tagging code --> def add_loss_op(self): """Defines the loss""" if self.config.use_crf: log_likelihood, trans_params = tf.contrib.crf.crf_log_likelihood( self.logits, self.labels, self.sequence_lengths) self.trans_params = trans_params # need to evaluate it for decoding self.loss = tf.reduce_mean(-log_likelihood)`

varshachawan commented 5 years ago
  1. This could be done once upon parameter initialization. parameter initialization of Tensorflow variables? Could you please elaborate.
guillaumegenthial commented 5 years ago

This is a very specific need that you need to address on your own. Googling for how to initialize Tensorflow variables from np array should help. But probably just more data will help.