search

strongio / keras-bert

A simple technique to integrate BERT from tf hub to keras
258 stars 108 forks source link

Text classification (keras_text): 'Node' object has no attribute 'output_masks' #23

Closed kerstenj closed 5 years ago

kerstenj commented 5 years ago

Hi,

i would like to use the output of a BertLayer as Input for a YookKimCNN. This is implemented in keras_text. I already realized that mixing tf and keras imports is not a good idea. However, the YoonKimCNN is currently "only" available in keras_text, leading to the following error:

AttributeError: 'Node' object has no attribute 'output_masks'

After several changes, I currently use tensorflow 1.12.0 and keras 2.2.4.

Thanks for any suggestions in advance

kerstenj commented 5 years ago

OK, so far, I managed to switch to keras imports + I realized that the output from a BertLayer needs to be expanded, e.g. by

bert_output = tf.expand_dims(bert_output, axis=-1)

This fails when I instantiate the model

self.model = Model(inputs = bert_inputs, outputs=all_outputs)

AttributeError: 'NoneType' object has no attribute '_inbound_nodes'

As stated here, wrapping using a Lambda layer should fix this issue. Since I am not familiar with this, I am facing a problem:

The shape of the bert result tensor is shape=(?, 768) When I try to expand this to (?,768,1) with

bert_output = Lambda(lambda x: expand_dims(x, -1))(bert_output)

I get the following error: Error converting shape to a TensorShape: int() argument must be a string, a bytes-like object or a number, not 'tuple'.

Thanks for any comments on this.

kerstenj commented 5 years ago

All right, it seems that using a Reshape layer does the trick:

    bert_output = BertLayer.BertLayer(n_fine_tune_layers=10)(bert_inputs) 
    bert_output_r = Reshape((768, 1))(bert_output)
    vanilla_model = YoonKimCNN(128, [3,4,5])