TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

[micosacak]

I am trying to have a model that can get different input_shapes. I use a (128,128,1) input_shape for for training data as all my training images have the same size.

However, after training the image size will be variable (e.g.; 312x256, 550x2300, etc). How to create a model that can get variable input shapes. The model below raises error in get_crop_shape, if I use input_shape as [None, None,1]. I have to use get_crop_shape to prevent problems during concatenations as tensor shape will be different if the shape is not multiple of 128.

if I use any shape excepts None (e.g.; [128, 128, 1] or [129, 239, 1]), the model works!

I need the model accepts different input_shapes, so I can use it in tensorflow/java.

In Python, I create model, perform training and then save the model as model.save("model"). Then, I create a new model based on the shape of new images and only load weights and it works! For instance, if I save my model after training as model.save("model"). I can use it for different image shapes as below.

import tensorflow as tf
model1 = Unet(input_shape = (129,239,1))
model1.load_weights("model/variables/variables")

model2= Unet(input_shape = (2300,3450,1))
model2.load_weights("model/variables/variables")

prediction1 = model1(input_tensor1) # input_tensor1 shape is (None, 129,239,1), None is the batch size.
prediction1 = model2(input_tensor2) # input_tensor2 shape is (None, 2300,3450,1), None is the batch size.

I ask the same question on tensorflow/java as well. #288

import os
import skimage.io as io
import cv2
import numpy as np
import tensorflow as tf
import time
import imageio
import matplotlib.pyplot as plt
import copy as copy
import matplotlib

def get_crop_shape(target, query):
    # the height
    channelHeight = target.get_shape()[1] - query.get_shape()[1]
    assert (channelHeight >= 0)
    channelHeight1 = int(channelHeight/2)
    if channelHeight % 2 != 0:
        channelHeight2 = channelHeight1 + 1
    else:
        channelHeight2 = channelHeight1
    # the width
    channelWidth = target.get_shape()[2] - query.get_shape()[2]
    assert (channelWidth >= 0)
    channelWidth1 = int(channelWidth/2)
    if channelWidth % 2 != 0:
        channelWidth2 = channelWidth1 + 1
    else:
        channelWidth2 = channelWidth1
    return (channelHeight1, channelHeight2), (channelWidth1, channelWidth2)

def getAct(x):
    return tf.keras.layers.ReLU()(x)

def Unet(input_shape = (None,None,1), kernelSize = 3, drop_level = 0.5, nChannels = 1):
    inputs = tf.keras.layers.Input(shape = [input_shape[0], input_shape[1], input_shape[2]])
    conv1 = tf.keras.layers.Conv2D(64, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(inputs)
    conv1 =  getAct(conv1)
    conv1 = tf.keras.layers.Conv2D(64, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv1)
    conv1 =  getAct(conv1)
    #drop1 = tf.keras.layers.Dropout(drop_level)(conv1)
    pool1 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(conv1) # drop1 --> conv1
    #
    conv2 = tf.keras.layers.Conv2D(128, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(pool1)
    conv2 =  getAct(conv2)
    conv2 = tf.keras.layers.Conv2D(128, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv2)
    conv2 =  getAct(conv2)
    #drop2 = tf.keras.layers.Dropout(drop_level)(conv2)
    pool2 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(conv2) #drop2 --> conv2
    #
    conv3 = tf.keras.layers.Conv2D(256, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(pool2)
    conv3 =  getAct(conv3)
    conv3 = tf.keras.layers.Conv2D(256, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv3)
    conv3 =  getAct(conv3)
    #drop3 = tf.keras.layers.Dropout(drop_level)(conv3)
    pool3 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(conv3)  #drop3 --> conv3
    #
    conv4 = tf.keras.layers.Conv2D(512, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(pool3)
    conv4 =  getAct(conv4)
    conv4 = tf.keras.layers.Conv2D(512, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv4)
    conv4 =  getAct(conv4)
    drop4 = tf.keras.layers.Dropout(drop_level)(conv4)
    pool4 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(drop4)
    #
    conv5 = tf.keras.layers.Conv2D(1024, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(pool4)
    conv5 =  getAct(conv5)
    conv5 = tf.keras.layers.Conv2D(1024, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv5)
    conv5 =  getAct(conv5)
    drop5 = tf.keras.layers.Dropout(drop_level)(conv5)
    up6 = tf.keras.layers.Conv2DTranspose(512, kernelSize, strides = (2,2), padding = 'same', kernel_initializer = 'he_normal')(drop5)
    ch, cw = get_crop_shape(drop4, up6)
    up6 = tf.keras.layers.ZeroPadding2D(padding=((ch[0], ch[1]), (cw[0], cw[1])))(up6) # add zeropadding.
    merge6 = tf.keras.layers.concatenate([drop4,up6], axis = 3)
    #
    conv6 = tf.keras.layers.Conv2D(512, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(merge6)
    conv6 =  getAct(conv6)
    conv6 = tf.keras.layers.Conv2D(512, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv6)
    conv6 =  getAct(conv6)
    up7 = tf.keras.layers.Conv2DTranspose(256, kernelSize, strides = (2,2), padding = 'same', kernel_initializer = 'he_normal')(conv6)
    ch, cw = get_crop_shape(conv3, up7)   #drop3 --> conv3
    up7 = tf.keras.layers.ZeroPadding2D(padding=((ch[0], ch[1]), (cw[0], cw[1])))(up7) # add zeropadding.
    merge7 = tf.keras.layers.concatenate([conv3,up7], axis = 3)   #drop3 --> conv3
    #
    conv7 = tf.keras.layers.Conv2D(256, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(merge7)
    conv7 =  getAct(conv7)
    conv7 = tf.keras.layers.Conv2D(256, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv7)
    conv7 =  getAct(conv7)
    up8 = tf.keras.layers.Conv2DTranspose(128, kernelSize, strides = (2,2), padding = 'same', kernel_initializer = 'he_normal')(conv7)
    ch, cw = get_crop_shape(conv2, up8) #drop2 --> conv2
    up8 = tf.keras.layers.ZeroPadding2D(padding=((ch[0], ch[1]), (cw[0], cw[1])))(up8) # add zeropadding.
    merge8 = tf.keras.layers.concatenate([conv2,up8], axis = 3) #drop2 --> conv2
    #
    conv8 = tf.keras.layers.Conv2D(128, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(merge8)
    conv8 =  getAct(conv8)
    conv8 = tf.keras.layers.Conv2D(128, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv8)
    conv8 =  getAct(conv8)
    up9 = tf.keras.layers.Conv2DTranspose(64, kernelSize, strides = (2,2), padding = 'same', kernel_initializer = 'he_normal')(conv8)
    ch, cw = get_crop_shape(conv1, up9) #drop1 --> conv1
    up9 = tf.keras.layers.ZeroPadding2D(padding=((ch[0], ch[1]), (cw[0], cw[1])))(up9) # add zeropadding.
    merge9 = tf.keras.layers.concatenate([conv1,up9], axis = 3) #drop1 --> conv1
    #
    conv9 = tf.keras.layers.Conv2D(64, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(merge9)
    conv9 =  getAct(conv9)
    conv9 = tf.keras.layers.Conv2D(64, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv9)
    conv9 =  getAct(conv9)
    conv9 = tf.keras.layers.Conv2D(2, kernelSize, padding = 'same', kernel_initializer = 'he_normal')(conv9)
    conv9 =  getAct(conv9)
    conv10 = tf.keras.layers.Conv2D(nChannels, 1, activation = 'sigmoid')(conv9)
    model = tf.keras.Model(inputs = inputs, outputs = conv10)
    model.compile(optimizer = tf.keras.optimizers.Adam(1e-4, beta_1 = 0.9, beta_2 = 0.999), loss = 'binary_crossentropy', metrics = ['accuracy'])
    print("Using UnetLeakyPReLU ...")
    print(model.summary())
    return model

[AdityaKane2001]

@micosacak Can you please share the error trace? Thanks

[micosacak]

@AdityaKane2001 Here is the output.

>>>my_model = Unet()
2021-04-14 17:32:12.327108: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-04-14 17:32:12.327370: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 37, in Unet
  File "<stdin>", line 3, in get_crop_shape
TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

[AdityaKane2001]

@micosacak This may be stupid on my part, but can you resize the inputs and use input_shape= (128,128,1)? Resize layer is available in keras.

[amahendrakar]

@micosacak, On running the code, I am facing an error stating NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for model/variables/variables. Please find the gist of it here.

In order to reproduce the issue reported, could you please provide the TensorFlow version, a minimal code snippet and the supporting files necessary to run the code.

Also, take a look at @AdityaKane2001's comment and check if it helps.

Thanks!

[micosacak]

@amahendrakar

model1 = Unet(input_shape = (129,239,1))
model1.load_weights("model/variables/variables")

model1.load_weights("model/variables/variables") will raise error of course. Because you do not have a trained model saved as "model" in the path. If you want any dataset you can download an example from here. But my question does not require an input data, I think. Because it is a more general question.

I think you did not get my question.

My question is:

How to generate a model that can get different input shapes, even after training? For instance, if I use (128,128,1) as input shape and save the model, the model requires to have always (128,128,1) input shape. If the image shape is different from (128,128,1), then it raises error. If I use (None, None,1) as input shape, then I can not initiate a model, as it raises this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 37, in Unet
  File "<stdin>", line 3, in get_crop_shape
TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

[micosacak]

@AdityaKane2001 what do you mean by resizing? Is this what you are suggesting?

[micosacak]

BTW; the tensorflow 2.4.1 raises the error, if the input shape is (256,256,1) as below:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ilias/py37CPUtf241/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 998, in __call__
    input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
  File "/home/ilias/py37CPUtf241/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_spec.py", line 274, in assert_input_compatibility
    ', found shape=' + display_shape(x.shape))
ValueError: Input 0 is incompatible with layer model_1: expected shape=(None, 128, 128, 1), found shape=(1, 256, 256, 1)

Now, if I try the same with tensorflow 2.2

WARNING:tensorflow:Model was constructed with shape (None, 128, 128, 1) for input Tensor("input_2:0", shape=(None, 128, 128, 1), dtype=float32), but it was called on an input with incompatible shape (1, 256, 256, 1).

However, tensorflow 2.2 still perform image segmentation correctly!

[micosacak]

In order to prevent above errors; I do the following (as I have shown above), if the image shape is (256,256,1):

import tensorflow as tf
model1 = Unet(input_shape = (256,256,1))
model1.load_weights("model/variables/variables")

then it does not show any error or warning message!

[AdityaKane2001]

@micosacak Yes, I was suggesting Resizing layer. The thing is, when you pass some integer values in shape, you essentially create a graph of the model. All the functions used in the model are now traced by AutoGraph. This means that your input size is now fixed. Thus, using None in your input shape means that you are not allowing the model to be traced. Also, when you compile the model, the model now fixes ambiguous shapes of the tensors that are inside the graph. Obviously, this cannot be done when you pass None . Hence the function get_crop_shape fails because while tracing a dummy tensor is passed in that function. @amahendrakar please correct me if I am wrong. Hope this helps.

[jvishnuvardhan]

@micosacak I think there are two options.

1. As @AdityaKane2001 mentioned, you can write your own preprocessing code that include reshaping the input image size to the size you have used in your training (say it as target_size), or
2. simply use ImageDataGenerator class that reshapes your input images to target_size as shown in this tutorial.

In the above tutorial there are several models that were trained with different size. Later those models were used with flower_dataset that is of different size when compared to the original images that were used for training those models. Hope it helps. Thanks!

[micosacak]

Thanks for your help, I will check the tutorial as well.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No