Missing 'inbound_nodes' in CRF layer with Theano-1.0.1 & Keras Keras-2.1.3

[Koziev]

Trying to run conll2000_chunking_crf.py with no modification of repo code on Python 2.7.13, Keras Keras-2.1.3 and Theano-1.0.1+30.g8aaf55bfc (bleeding edge versions for both), throws the following error:

Traceback (most recent call last):
  File "conll2000_chunking_crf.py", line 65, in <module>
    model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy])
  File "C:\Users\eek\Anaconda2\lib\site-packages\keras\models.py", line 826, in compile
    **kwargs)
  File "C:\Users\eek\Anaconda2\lib\site-packages\keras\engine\training.py", line 827, in compile
    sample_weight, mask)
  File "C:\Users\eek\Anaconda2\lib\site-packages\keras\engine\training.py", line 426, in weighted
    score_array = fn(y_true, y_pred)
  File "build\bdist.win-amd64\egg\keras_contrib\layers\crf.py", line 287, in loss
AttributeError: 'CRF' object has no attribute 'inbound_nodes'

[FredRodrigues]

I solved this problem by downgrade the version of keras to 2.1.

[Koziev]

I solved this problem by downgrade the version of keras to 2.1.

Could you tell the proper version of Theano? I've tried Keras 2.1.0 and Theano 1.0.1:

(keras-contrib-2.1) (base) e:\polygon\keras-contrib-2.1\keras-contrib\examples>python -c "import keras; print(keras.__version__)"
Using Theano backend.
WARNING (theano.configdefaults): install mkl with `conda install mkl-service`: No module named mkl
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
2.1.0

(keras-contrib-2.1) (base) e:\polygon\keras-contrib-2.1\keras-contrib\examples>pip show theano
Name: Theano
Version: 1.0.1
Summary: Optimizing compiler for evaluating mathematical expressions on CPUs and GPUs.
Home-page: http://deeplearning.net/software/theano/
Author: LISA laboratory, University of Montreal
Author-email: theano-dev@googlegroups.com
License: BSD
Location: e:\polygon\keras-contrib-2.1\lib\site-packages
Requires: six, scipy, numpy

And got another error when running conll2000_chunking_crf.py:

==== training CRF ====
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
embedding_1 (Embedding)      (None, None, 200)         1787200
_________________________________________________________________
crf_1 (CRF)                  (None, None, 23)          5198
=================================================================
Total params: 1,792,398
Trainable params: 1,792,398
Non-trainable params: 0
_________________________________________________________________
Traceback (most recent call last):
  File "conll2000_chunking_crf.py", line 66, in <module>
    model.fit(train_x, train_y, epochs=EPOCHS, validation_data=[test_x, test_y])
  File "e:\polygon\keras-contrib-2.1\lib\site-packages\keras\models.py", line 960, in fit
    validation_steps=validation_steps)
  File "e:\polygon\keras-contrib-2.1\lib\site-packages\keras\engine\training.py", line 1572, in fit
    batch_size=batch_size)
  File "e:\polygon\keras-contrib-2.1\lib\site-packages\keras\engine\training.py", line 1411, in _standardize_user_data
    exception_prefix='target')
  File "e:\polygon\keras-contrib-2.1\lib\site-packages\keras\engine\training.py", line 153, in _standardize_input_data
    str(array.shape))
ValueError: Error when checking target: expected crf_1 to have shape (None, None, 23) but got array with shape (8936L, 7
8L, 1L)

[FredRodrigues]

I'm currently using tensorflow as backend. Regarding the error, have you changed the code ?

[Koziev]

Regarding the error, have you changed the code ?

No, I have not. I run unmodified conll2000_chunking_crf.py.

[emirceyani]

@FredRodrigues it really worked. Thanks a lot!

[Shnurre]

@FredRodrigues downgrading keras to 2.1.0 does indeed help the model to compile. Thank you a lot!

However I'm getting the same error as @Koziev had described above: "ValueError: Error when checking target: expected crf_1 to have shape (None, None, 23) but got array with shape (8936L, 7 8L, 1L)" Looks like there are some problems with sparsity even when using CRF with "sparse_target=True". Indeed, when having sparse targets, we are supposed to have the last shape = 1, so the error seems to fire in a place it shouldn't (and if I use "sparse_target=False" I am getting the same error).

I used both theano and tensorflow backend and had the same error on both. Can anyone confirm that there is indeed some problem with sparse targets on keras 2.1.0?

[FredRodrigues]

Hey. I've just run the exact code from conll2000_chunking_crf.py file and haven't got any problem.

I don't know if this is the problem but actually i'm running Keras 2.0.6. Try use the following versions:

Keras==2.0.6 keras-contrib==2.0.8 tensorflow-gpu==1.4.1

[Shnurre]

@FredRodrigues Thanks a lot, man! Downgrading keras to 2.0.6 did solve this problem for me. Now conll2000_chunking_crf.py (as well as my own code which uses crf with sparse targets) works as it supposed to.

[Koziev]

@FredRodrigues thank you for working solution!

Downgrading to keras==2.0.6 and switching to Tensorflow helps me. KERAS_BACKEND=theano does not work (see https://github.com/keras-team/keras-contrib/issues/131).

[TIANRENK]

@FredRodrigues I already have this problem? My following versions: keras==2.0.6 keras-contrib==2.0.8 tensorflow without gpu ==1.4.0 Could you tell me how to solve this problem?

[FredRodrigues]

@TIANRENK If you have the same versions as me I don't see what is causing the issue. But try looking at #219

[tarmeens]

Unfortunately I have the same problem:

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, **kwargs)
    838             loss_weight = loss_weights_list[i]
    839             output_loss = weighted_loss(y_true, y_pred,
--> 840                                         sample_weight, mask)
    841             if len(self.outputs) > 1:
    842                 self.metrics_tensors.append(output_loss)

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in weighted(y_true, y_pred, weights, mask)
    444         """
    445         # score_array has ndim >= 2
--> 446         score_array = fn(y_true, y_pred)
    447         if mask is not None:
    448             # Cast the mask to floatX to avoid float64 upcasting in theano

/usr/local/lib/python2.7/dist-packages/keras_contrib/layers/crf.pyc in loss(y_true, y_pred)
    285         if self.learn_mode == 'join':
    286             def loss(y_true, y_pred):
--> 287                 assert self._inbound_nodes, 'CRF has not connected to any layer.'
    288                 assert not self._outbound_nodes, 'When learn_model="join", CRF must be the last layer.'
    289                 if self.sparse_target:

AttributeError: 'CRF' object has no attribute '_inbound_nodes'

My versions: keras == 2.0.6 (also tried with 2.1.1) keras-contrib = latest (2.0.8) tensorflow no gpu 1.4.0

[FredRodrigues]

Hum.. I'm using python3.

[miguendes]

On my desktop I tried keras==2.1.2 and it worked. On my laptop it doesn't, then I decided to upgrade keras to the lastest version so far (2.1.4) and it worked. I'm still puzzled why it works on my desktop but doesn't on my laptop. To make sure my env is the same I'm isolating the packages using virtualenv.

[tarmeens]

@FredRodrigues @mendesmiguel I'm working on a laptop. I tried keras==2.1.1 and tf ==1.4.0 with Python 3.6.1 on Windows and it didn't work. I then updated keras to the latest version (2.1.4) and now it seems it is working. Thanks for yor help!

[stephen-v]

@FredRodrigues I'm working . I tried keras =2.1.4 and tf ==1.4.0 with python 3.6.2 on Windows 10

[find-fun]

@FredRodrigues I'm working. Firstly. I tried keras 2.06, 2.08, 2.1.0, 2.1.6, and they all didn't work. Finally, I find the tenflow wrong! When I upgrade the tensorflow to 1.8.0, and with keras2.1.6, it works! Thanks!

[lanjinglingtamei]

I have solved the problem, update your keras-contrib to the latest version, my tensorflow == 1.11.0, keras == 2.2.4