search

keras-team / keras-contrib

Keras community contributions
MIT License
1.58k stars 650 forks source link

CRF error #507

Open ly2014 opened 5 years ago

ly2014 commented 5 years ago

Since the CRF layer does not change the output shape of the network, why can't the dense layer be connected directly behind it?

my code： model.add(Bidirectional(GRU(32, return_sequences=True))) model.add(Dropout(0.5)) model.add(CRF(64)) model.add(Dense(1, activation='sigmoid'))

helpmefindaname commented 5 years ago

It can be. It would be nice if you post the error you get.

ly2014 commented 5 years ago

If I connect Dense directly behind the CRF layer, the following error will occur。

  File "C:\Program Files\Python36\lib\site-packages\keras_contrib-2.0.8-py3.6.egg\keras_contrib\losses\crf_losses.py", line 55, in crf_loss
AttributeError: 'Dense' object has no attribute 'learn_mode'

And I have another question. I use CRF model to train text classifier and assign the same category to every word in a sentence, but why are the words in each sentence not exactly the same in the predicted value?

helpmefindaname commented 5 years ago

This happens when you use the crf loss. The crf loss is (like stated in the documentation) only for cases where the crf is the last layer. Use crossentropy loss instead. Also you might want to set the learnmode to marginalized.

To the latter: because in neural networks, the predicted value is not the same as the expected one. It lerns what it should predict but that doesn't mean that it can generalize. You could try with a larger hidden size or more layers to get better results.