search

guillaumegenthial / tf_ner

Simple and Efficient Tensorflow implementations of NER models with tf.estimator and tf.data
Apache License 2.0
923 stars 275 forks source link

couldn't get F1 91.21 #33

Open mungg opened 5 years ago

mungg commented 5 years ago

Thanks for your clear code and instruction! I run a code on CONLL2003 without modifying other hyper-parameter and structure, the F1 score of testb is about to 90.6 which is not reaching the your report(91.2) (at char CNN + LSTM+CRF) also, LSTM+CRF shows the 89.xx~ 90.0x. I got a little bit lower scores comparing with your reports. I was wondering if it needed to change the default hyper-parameter.
Could your provide some insight about this result?

lanzhuzhu commented 5 years ago

I met the same problem. I used the embedding "glove.6B.300d.txt" and tested the two versions of your code. One model is "chars_lstm_lstm_crf model ", the other is "sequence_tagging". From my results, the chars_lstm_lstm_crf model performs worse in the same condition. Is this common?

guillaumegenthial commented 5 years ago

@mungg and @lanzhuzhu , Try using the 840B.300d vectors as well as the BIOES tagging scheme. You should be able to get good results (in average). Note that some runs perform better than others and the results I report on the README are averaged over 5 runs.