[ISSUE][HOW TO FIX] InvalidArgumentError: cannot compute MatMul as input #0(zero-based) was expected to be a float tensor but is a uint8 tensor [Op:MatMul]

[ScGPS]

@michelucci , Your book is perfect! Thank you!

Here, I found a issue, I have fixed it. I posted the fixed method. In tensorflow 1.13.1, I run your code, a issue encountered.

Original Code

import tensorflow as tf
import keras
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('default')

import tensorflow.contrib.eager as tfe

#from google.colab import files

tf.enable_eager_execution()

num_classes = 10

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

image_vector_size = 28*28
x_train = x_train.reshape(x_train.shape[0], image_vector_size).astype('float32')
x_test = x_test.reshape(x_test.shape[0], image_vector_size).astype('float32')

y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

dataset = tf.data.Dataset.from_tensor_slices(
    (tf.cast(x_train/255.0, tf.float32),
     tf.cast(y_train,tf.int64)))

dataset = dataset.shuffle(60000).batch(64)

mnist_model = tf.keras.Sequential([
  tf.keras.layers.Dense(1024, input_shape=(784,)),  # must declare input shape
  tf.keras.layers.Dense(1024),  # must declare input shape
  tf.keras.layers.Dense(10)
])

optimizer = tf.train.AdamOptimizer()
loss_history = []

#%%time
with tf.device('/cpu:0'):
  for i in range(1): # Loop for the Epochs
    print ("\nEpoch:", i)
    for (batch, (images, labels)) in enumerate(dataset.take(60000)): # Loop for the mini-batches
      if batch % 100 == 0:
        print('.', end='')
      with tf.GradientTape() as tape:
        logits = mnist_model(images, training=True)
        loss_value = tf.losses.sparse_softmax_cross_entropy(tf.argmax(labels, axis = 1), logits)

        loss_history.append(loss_value.numpy())
        grads = tape.gradient(loss_value, mnist_model.variables)
        optimizer.apply_gradients(zip(grads, mnist_model.variables),
                                  global_step=tf.train.get_or_create_global_step())

probs = tf.nn.softmax(mnist_model(x_train))
print(tf.reduce_mean(tf.cast(tf.equal(tf.argmax(probs, axis=1), tf.argmax(y_train, axis = 1)), tf.float32)))

Output

Epoch: 0
..........
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-4-6753e0f3f94f> in <module>
     54                                   global_step=tf.train.get_or_create_global_step())
     55 
---> 56 probs = tf.nn.softmax(mnist_model(x_train))
     57 print(tf.reduce_mean(tf.cast(tf.equal(tf.argmax(probs, axis=1), tf.argmax(y_train, axis = 1)), tf.float32)))

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    590       else:
    591         # Eager execution on data tensors.
--> 592         outputs = self.call(inputs, *args, **kwargs)
    593         self._handle_activity_regularization(inputs, outputs)
    594         return outputs

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\tensorflow\python\keras\engine\sequential.py in call(self, inputs, training, mask)
    228   def call(self, inputs, training=None, mask=None):
    229     if self._is_graph_network:
--> 230       return super(Sequential, self).call(inputs, training=training, mask=mask)
    231 
    232     outputs, _ = self._call_and_compute_mask(

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\tensorflow\python\keras\engine\network.py in call(self, inputs, training, mask)
    813     outputs, _ = self._run_internal_graph(inputs,
    814                                           training=training,
--> 815                                           mask=masks)
    816     return outputs
    817 

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\tensorflow\python\keras\engine\network.py in _run_internal_graph(self, inputs, training, mask)
    998               else:
    999                 if context.executing_eagerly():
-> 1000                   output_tensors = layer(computed_tensor, **kwargs)
   1001                 else:
   1002                   output_tensors = layer.call(computed_tensor, **kwargs)

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    590       else:
    591         # Eager execution on data tensors.
--> 592         outputs = self.call(inputs, *args, **kwargs)
    593         self._handle_activity_regularization(inputs, outputs)
    594         return outputs

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\tensorflow\python\keras\layers\core.py in call(self, inputs)
    973         outputs.set_shape(output_shape)
    974     else:
--> 975       outputs = gen_math_ops.mat_mul(inputs, self.kernel)
    976     if self.use_bias:
    977       outputs = nn.bias_add(outputs, self.bias)

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\tensorflow\python\ops\gen_math_ops.py in mat_mul(a, b, transpose_a, transpose_b, name)
   5321       else:
   5322         message = e.message
-> 5323       _six.raise_from(_core._status_to_exception(e.code, message), None)
   5324   # Add nodes to the TensorFlow graph.
   5325   if transpose_a is None:

D:\ProgramData\Anaconda3\envs\tensorflow-gpu-keras\lib\site-packages\six.py in raise_from(value, from_value)

InvalidArgumentError: cannot compute MatMul as input #0(zero-based) was expected to be a float tensor but is a uint8 tensor [Op:MatMul]

How to fix

Modify

x_train = x_train.reshape(x_train.shape[0], image_vector_size)
x_test = x_test.reshape(x_test.shape[0], image_vector_size)

To

x_train = x_train.reshape(x_train.shape[0], image_vector_size).astype('float32')
x_test = x_test.reshape(x_test.shape[0], image_vector_size).astype('float32')

After fixed, Output

Epoch: 0
..........tf.Tensor(0.88195, shape=(), dtype=float32)

[michelucci]

Thanks a lot @ScGPS for checking this with the old version of TF! Greatly appreciated. Best, Umberto

[ScGPS]

@michelucci, I followed your codes in your book. The codes can run correctly in tensorflow 1.13.1. There are some codes that can't run correctly in tensorflow 1.15.0.

implicit_gradients is not supported in 1.15.0.

grad = tfe.implicit_gradients(loss)

Your book is very profound. At any time, this book can deeply reveal the essence of deep learning, namely regression calculation. Thank you for your contribution!