AttributeError: The layer has never been called and thus has no defined input shape

[shishaozheng]

I train a model using CNN and BLSTM with keras 2.0.8 using tensorflow as backend. The "AttributeError: The layer has never been called and thus has no defined input shape" occurred when converting to coreml model.

keras model
    input = Input(shape=(height,None,1), name='the_input')
    #conv layers
    ...
    ...
    m = TimeDistributed(Flatten(),name='timedistrib')(m)

    m = Bidirectional(LSTM(rnnunit,return_sequences=True),name='blstm1')(m)
    m = Dense(rnnunit,name='blstm1_out',activation='linear')(m)
    m = Bidirectional(LSTM(256,return_sequences=True),name='blstm2')(m)
    output= Dense(nclass,name='blstm2_out',activation='softmax')(m)

    basemodel = Model(inputs=input,outputs=output)

convert method:
    scale = 1./255
    coreml_model = coremltools.converters.keras.convert(keras_model_path,
                                                    input_names=['the_input'],
                                                    output_names=['blstm2_out'],
                                                    image_input_names=['the_input'],
                                                    image_scale=scale)

[aseemw]

There seems to be a bug in the time distributed flatten layer. Meanwhile, can you try converting the model till the flatten layer and a second model starting from the the bidirectional layer.

[teaglin]

@aseemw Can you provide more detail on exporting separate parts of the model? There seems to be no direct way to do this with the API and specifying the input and output names doesn't do anything. The same error persists.

[aseemw]

input_model1 = Input(shape=(height,width,1), name='the_input')
    #conv layers
    ...
    ...
out_model1 = last_layer
model1 = Model(inputs=input_model1,outputs=out_model1)

input_model2 = Input(shape=model1.output_shape, name='the_input2')
    #bidir layers
    ...
    ...
out_model2 = last_layer
model2 = Model(inputs=input_model2,outputs=out_model2)

using get_weights() and set_weights() functions in Keras, you can copy the weights from the trained (single) keras model to these two models and then convert them separately to CoreML.

[teaglin]

Thanks for the response. I tried your suggestion and I am getting stuck with the input into the LSTM layer. It gives the following error – ValueError: Input 0 is incompatible with layer lstm: expected ndim=3, found ndim=4

But I am feeding the output from model 1 directly into the layer. If I remove the second model and chain it together everything works otherwise. I've included the keras model.

model.add(TimeDistributed(Convolution2D(16, 3, 3, border_mode="same", activation='relu'), input_shape=(None, image_height, image_width, 3)))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Convolution2D(32, 3, 3, activation='relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Convolution2D(64, 3, 3, activation='relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Convolution2D(128, 3, 3, activation='relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Flatten(), name='out1'))

input_model2 = Input(shape=model.output_shape, name='the_input2')
model2 = LSTM(256, name='lstm')(input_model2)
model2 = Dense(1, activation='sigmoid', name='out')(model2)

[teaglin]

@aseemw I was able to figure out a way around it. While not the best solution. The model trains and the solution converges well. I was able to export the model to CoreML, but I can't seem to get the correct solution compared to Keras.

for x in range(length):
        x = Input(shape=(image_height, image_width, 3))
        cnn = Convolution2D(16, 3, 3, border_mode="same", activation='relu')(x)
        cnn = MaxPooling2D(pool_size=(2, 2))(cnn)

        cnn = Convolution2D(32, 3, 3, border_mode="same", activation='relu')(cnn)
        cnn = MaxPooling2D(pool_size=(2, 2))(cnn)

        cnn = Convolution2D(64, 3, 3, border_mode="same", activation='relu')(cnn)
        cnn = MaxPooling2D(pool_size=(2, 2))(cnn)

        cnn = Convolution2D(128, 3, 3, border_mode="same", activation='relu')(cnn)
        cnn = MaxPooling2D(pool_size=(2, 2))(cnn)

        cnn = Flatten()(cnn)

        nets.append(cnn)
        inputs.append(x)

merged = concatenate(nets)
merged = Reshape((length, 25088))(merged)
out = LSTM(256, name='lstm')(merged)
out = Dense(1, activation='sigmoid')(out)
t = Model(inputs, out)

These are the parameters for prediction. Do I need to populate lstm_h_in & lstm_c_in?

/**
        Make a prediction using the convenience interface
        - parameters:
            - image1 as color (kCVPixelFormatType_32BGRA) image buffer, 225 pixels wide by 225 pixels high
            - image2 as color (kCVPixelFormatType_32BGRA) image buffer, 225 pixels wide by 225 pixels high
            - image3 as color (kCVPixelFormatType_32BGRA) image buffer, 225 pixels wide by 225 pixels high
            - image4 as color (kCVPixelFormatType_32BGRA) image buffer, 225 pixels wide by 225 pixels high
            - lstm_h_in as optional 256 element vector of doubles
            - lstm_c_in as optional 256 element vector of doubles
        - throws: an NSError object that describes the problem
        - returns: the result of the prediction as output
    */

[TobyRoseman]

Multi-backend Keras support has been removed in coremltools 6.0.