using Elmo embedding layer in image captioning model ( Tutorial #22)

Hello, Thanks for the tutorial. I want to use Elmo embedding method replaced the embedding layer that you used in your image captioning proposed model, but it doesn't work. Elmo needs a string input so I changed the input type from float32 to string and then add an Elmo layer replaced embedding layer. I would be grateful if you can help me with how to use Elmo in your model. thank you in advance Shima

input: decoder_input =Input(shape=(None,), dtype="string", name='decoder_input')

layer in the model: net=ElmoEmbeddingLayer()(decoder_input)

Elmo Function:

class ElmoEmbeddingLayer(Layer):

def __init__(self, **kwargs):
    self.dimensions = 1024
    self.trainable=True
    super(ElmoEmbeddingLayer, self).__init__(**kwargs)

def build(self, input_shape):
    self.elmo = hub.Module('/TensorFlow-Tutorials-master/coco/2', trainable=self.trainable,
                           name="{}_module".format(self.name))

    self.trainable_weights +=K.tf.trainable_variables(scope="^{}_module/.*".format(self.name))
    super(ElmoEmbeddingLayer, self).build(input_shape)

def call(self, x, mask=None):
    result = self.elmo(K.squeeze(K.cast(x, tf.string), axis=1),
                  as_dict=True,
                  signature='default',
                  )['default']
    return result

def compute_mask(self, inputs, mask=None):
    return K.not_equal(inputs, '--PAD--')

def compute_output_shape(self, input_shape):
    return (input_shape[0],input_shape[1], self.dimensions)

Error ValueError: Input 0 is incompatible with layer decoder_gru1: expected ndim=3, found ndim=2

Hvass-Labs / TensorFlow-Tutorials

using Elmo embedding layer in image captioning model ( Tutorial #22) #112