Error in model.load_weights using Keras 2.0.4

[bigdata2]

Build model... 2017-05-15 23:00:29.615498: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-05-15 23:00:29.615533: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. deepnet_predict.py:92: UserWarning: Update your Conv1D call to the Keras 2 API: Conv1D(activation="relu", padding="valid", strides=1, filters=64, kernel_size=5) subsample_length=1)) deepnet_predict.py:99: UserWarning: Update your Conv1D call to the Keras 2 API: Conv1D(activation="relu", padding="valid", strides=1, filters=64, kernel_size=5) subsample_length=1)) deepnet_predict.py:118: UserWarning: Update your Conv1D call to the Keras 2 API: Conv1D(activation="relu", padding="valid", strides=1, filters=64, kernel_size=5) subsample_length=1)) deepnet_predict.py:125: UserWarning: Update your Conv1D call to the Keras 2 API: Conv1D(activation="relu", padding="valid", strides=1, filters=64, kernel_size=5) subsample_length=1)) deepnet_predict.py:134: UserWarning: The dropout argument is no longer support in Embedding. You can apply a keras.layers.SpatialDropout1D layer right after the Embedding layer to get the same behavior. model5.add(Embedding(len(word_index) + 1, 300, input_length=40, dropout=0.2)) deepnet_predict.py:135: UserWarning: Update your LSTM call to the Keras 2 API: LSTM(300, dropout=0.2, recurrent_dropout=0.2) model5.add(LSTM(300, dropout_W=0.2, dropout_U=0.2)) deepnet_predict.py:138: UserWarning: The dropout argument is no longer support in Embedding. You can apply a keras.layers.SpatialDropout1D layer right after the Embedding layer to get the same behavior. model6.add(Embedding(len(word_index) + 1, 300, input_length=40, dropout=0.2)) deepnet_predict.py:139: UserWarning: Update your LSTM call to the Keras 2 API: LSTM(300, dropout=0.2, recurrent_dropout=0.2) model6.add(LSTM(300, dropout_W=0.2, dropout_U=0.2)) deepnet_predict.py:142: UserWarning: The Merge layer is deprecated and will be removed after 08/2017. Use instead layers from keras.layers.merge, e.g. add, concatenate, etc. merged_model.add(Merge([model1, model2, model3, model4, model5, model6], mode='concat')) Traceback (most recent call last): File "deepnet_predict.py", line 179, in merged_model.load_weights('weights.h5') File "/home/anurag/anaconda2/lib/python2.7/site-packages/keras/models.py", line 717, in load_weights topology.load_weights_from_hdf5_group(f, layers) File "/home/anurag/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 2970, in load_weights_from_hdf5_group K.batch_set_value(weight_value_tuples) File "/home/anurag/anaconda2/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 2148, in batch_set_value assign_op = x.assign(assign_placeholder) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 512, in assign return state_ops.assign(self._variable, value, use_locking=use_locking) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 270, in assign validate_shape=validate_shape) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 47, in assign use_locking=use_locking, name=name) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op op_def=op_def) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2338, in create_op set_shapes_for_outputs(ret) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1719, in set_shapes_for_outputs shapes = shape_func(op) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1669, in call_with_requiring return call_cpp_shape_fn(op, require_shape_fn=True) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn debug_python_shape_fn, require_shape_fn) File "/home/anurag/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 676, in _call_cpp_shape_fn_impl raise ValueError(err.message) ValueError: Dimension 0 in both shapes must be equal, but are 57526 and 79197 for 'Assign_24' (op: 'Assign') with input shapes: [57526,300], [79197,300].

[ylmeng]

I have the same problem. I have a custom layer and cannot make keras.models.load_model work in any way I tried. So I have to use model.save_weights and model.load_weights. Unfortunately it does not work either. So now I cannot use my model which I trained for hours.

For load_weights, I got: /home/my_username/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.pyc in _call_cpp_shape_fn_impl(op, input_tensors_needed, input_tensors_as_shapes_needed, debug_python_shape_fn, require_shape_fn) 674 missing_shape_fn = True 675 else: --> 676 raise ValueError(err.message) 677 678 if missing_shape_fn:

ValueError: Dimension 0 in both shapes must be equal, but are 512 and 768 for 'Assign_21' (op: 'Assign') with input shapes: [512,4656], [768,5704].

[iperov]

@ylmeng did you solve problem ?

[ylmeng]

@iperov No. However there is a way to work around. You can load a raw model first, train for just 1 epoch, and then load your saved weights using the "trained" model. It is ugly but it works for me.

[iperov]

@ylmeng I solved problem. Custom layer must have get_config() override for serialize inner params thats are important

example of my BatchNorm2D custom layer that I ported from Torch to Keras

class TorchBatchNorm2D(keras.engine.topology.Layer):
    def __init__(self, axis=-1, momentum=0.99, epsilon=1e-3, **kwargs):
        super(TorchBatchNorm2D, self).__init__(**kwargs)
        self.supports_masking = True
        self.axis = axis
        self.momentum = momentum
        self.epsilon = epsilon

    def build(self, input_shape):
        dim = input_shape[self.axis]
        if dim is None:
            raise ValueError('Axis ' + str(self.axis) + ' of ' 'input tensor should have a defined dimension ' 'but the layer received an input with shape ' + str(input_shape) + '.')
        shape = (dim,)
        self.gamma = self.add_weight(shape=shape, name='gamma', initializer='ones', regularizer=None, constraint=None)
        self.beta = self.add_weight(shape=shape, name='beta', initializer='zeros', regularizer=None, constraint=None)
        self.moving_mean = self.add_weight(shape=shape, name='moving_mean', initializer='zeros', trainable=False)            
        self.moving_variance = self.add_weight(shape=shape, name='moving_variance', initializer='ones', trainable=False)            
        self.built = True

    def call(self, inputs, training=None):
        input_shape = K.int_shape(inputs)

        broadcast_shape = [1] * len(input_shape)
        broadcast_shape[self.axis] = input_shape[self.axis]

        broadcast_moving_mean = K.reshape(self.moving_mean, broadcast_shape)
        broadcast_moving_variance = K.reshape(self.moving_variance, broadcast_shape)
        broadcast_gamma = K.reshape(self.gamma, broadcast_shape)
        broadcast_beta = K.reshape(self.beta, broadcast_shape)        
        invstd = K.ones (shape=broadcast_shape, dtype='float32') / K.sqrt(broadcast_moving_variance + K.constant(self.epsilon, dtype='float32'))

        return (inputs - broadcast_moving_mean) * invstd * broadcast_gamma + broadcast_beta

    def get_config(self):
        config = { 'axis': self.axis, 'momentum': self.momentum, 'epsilon': self.epsilon }
        base_config = super(TorchBatchNorm2D, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

and loading works fine

keras_model = keras.models.load_model (model_path, custom_objects={'TorchBatchNorm2D': TorchBatchNorm2D} )