search

CyberZHG / keras-lr-multiplier

Learning rate multiplier
https://pypi.org/project/keras-lr-multiplier/
Other
46 stars 11 forks source link

InvalidArgumentError: cannot compute Equal as input #1(zero-based) was expected to be a float tensor but is a int64 tensor [Op:Equal] #4

Closed amitjslearn closed 4 years ago

amitjslearn commented 4 years ago

Here is the full code https://github.com/amitjslearn/Experiments/blob/master/kmnist-vgg-like.ipynb

Version = 0.8.0 I'm using the latest version - yes Python 3.7.4

Minimal Codes To Reproduce

model = load_model('vggnet5.h5')
epochs = 10
learning_rate=1e-2
mult_dict={}
dict_len=0
for layer in model.layers:
    if layer.trainable and layer.weights:
        dict_len+=1
lr=learning_rate
muls =[lr/(dict_len - i) for i in range(dict_len)]
muls = iter(muls)
for layer in model.layers:
    if layer.trainable and layer.weights:
        mult_dict[layer.name]=next(muls)

optimizer=LRMultiplier('adam', mult_dict)

model.compile(loss='categorical_crossentropy',
              optimizer=optimizer,
              metrics=['accuracy'])
learning_rate_reduction = ReduceLROnPlateau(monitor='val_loss', 
                                            patience=200,
                                            verbose=1,
                                            factor=0.2)

es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=200)

history = model.fit_generator(train_datagen.flow(x_train, y_train, batch_size=batch_size, shuffle=True),
                              steps_per_epoch=100,
                              epochs=epochs,
                              validation_data=valid_datagen.flow(x_valid, y_valid),
                              validation_steps=50,
                              callbacks=[learning_rate_reduction, es])

Here is the full code https://github.com/amitjslearn/Experiments/blob/master/kmnist-vgg-like.ipynb

The Error

{'block1_conv1': 0.0004347826086956522, 'batch_normalization_11': 0.00045454545454545455, 'block1_conv2': 0.0004761904761904762, 'batch_normalization_12': 0.0005, 'block2_conv1': 0.0005263157894736842, 'batch_normalization_13': 0.0005555555555555556, 'block2_conv2': 0.0005882352941176471, 'batch_normalization_14': 0.000625, 'block3_conv1': 0.0006666666666666666, 'batch_normalization_15': 0.0007142857142857143, 'block3_conv2': 0.0007692307692307692, 'batch_normalization_16': 0.0008333333333333334, 'block3_conv3': 0.0009090909090909091, 'batch_normalization_17': 0.001, 'block4_conv1': 0.0011111111111111111, 'batch_normalization_18': 0.00125, 'block4_conv2': 0.0014285714285714286, 'batch_normalization_19': 0.0016666666666666668, 'block4_conv3': 0.002, 'batch_normalization_20': 0.0025, 'fc-1': 0.0033333333333333335, 'fc-2': 0.005, 'predictions': 0.01} 0 [<tf.Variable 'Adam/iterations:0' shape=() dtype=int64, numpy=0>, <tf.Variable 'm_0:0' shape=(3, 3, 1, 32) dtype=float32, numpy=
array([[[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]]],

       [[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]]],

       [[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]]]], dtype=float32)>, <tf.Variable 'm_1:0' shape=(32,) dtype=float32, numpy=
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
      dtype=float32)>, <tf.Variable 'v_0:0' shape=(3, 3, 1, 32) dtype=float32, numpy=
array([[[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]]],

       [[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]]],

       [[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]],

        [[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0.]]]], dtype=float32)>, <tf.Variable 'v_1:0' shape=(32,) dtype=float32, numpy=
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
      dtype=float32)>, <tf.Variable 'vhat_0:0' shape=(1,) dtype=float32, numpy=array([0.], dtype=float32)>, <tf.Variable 'vhat_1:0' shape=(1,) dtype=float32, numpy=array([0.], dtype=float32)>]
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<timed exec> in <module>

~/anaconda3/lib/python3.7/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/anaconda3/lib/python3.7/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1730             use_multiprocessing=use_multiprocessing,
   1731             shuffle=shuffle,
-> 1732             initial_epoch=initial_epoch)
   1733 
   1734     @interfaces.legacy_generator_methods_support

~/anaconda3/lib/python3.7/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
     40 
     41     do_validation = bool(validation_data)
---> 42     model._make_train_function()
     43     if do_validation:
     44         model._make_test_function()

~/anaconda3/lib/python3.7/site-packages/keras/engine/training.py in _make_train_function(self)
    314                     training_updates = self.optimizer.get_updates(
    315                         params=self._collected_trainable_weights,
--> 316                         loss=self.total_loss)
    317                 updates = self.updates + training_updates
    318 

~/anaconda3/lib/python3.7/site-packages/keras_lr_multiplier/multiplier.py in get_updates(self, loss, params)
     78             print(self.multipliers, i, self.optimizer.weights)
     79             for w in self.optimizer.weights:
---> 80                 if w not in self.weights:
     81                     self.weights.append(w)
     82         setattr(self, self.lr_attr, origin_lr)

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py in __eq__(self, other)
   1096         return gen_math_ops.equal(self, other, incompatible_shape_error=False)
   1097       else:
-> 1098         return gen_math_ops.equal(self, other)
   1099     else:
   1100       # In legacy graph mode, tensor equality is object equality

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py in equal(x, y, incompatible_shape_error, name)
   3618       else:
   3619         message = e.message
-> 3620       _six.raise_from(_core._status_to_exception(e.code, message), None)
   3621   # Add nodes to the TensorFlow graph.
   3622   if incompatible_shape_error is None:

~/anaconda3/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: cannot compute Equal as input #1(zero-based) was expected to be a float tensor but is a int64 tensor [Op:Equal]

Here is the full code https://github.com/amitjslearn/Experiments/blob/master/kmnist-vgg-like.ipynb

stale[bot] commented 4 years ago

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

Nishitha-GK commented 4 years ago

Did you resolve this issue? I am looking for a possible solution too.