The predict image is black?

[frankdeepl]

Ｈｉ,guys,

I tried to use own data to train the model, and the data is RPG image. However, when I tried to predict test dataset, all of the result is black image, is there anybody know what is this problem?

[jiangang2018]

I have same problem. Anybody know what is this problem?

[jacobwhgordon]

I think there could probably be a lot of causes of this, but I was also getting all black predicted images for both colored and grayscale data I was trying to train on, while when I used the membrane data supplied by this repository the network was working.

I was able to fix my problem by adding batch normalization layers after every 2d convolution later (except the final one, 'conv10').

For example, the first block of layers becomes:

conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)
conv1 = BatchNormalization()(conv1)
conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)
conv1 = BatchNormalization()(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)

Hopefully, this will help someone else. Good luck everyone.

[JCuomo]

I struggled with the same issue of having black or grey images as outputs.

To be sure you are dealing with the same problem you can try this and see if you see something different than black:

#First, bypass the testGenerator function
# you should have in dir2 an input image named "image_something.png" and wahtever other image named "mask_something.png"
(x_test, y_test) = geneTrainNpy("dir2/","dir2/")
plt.subplot(1, 2, 1)
plt.imshow(x_test[0,:,:,0])
plt.subplot(1, 2, 2)
predictions = model.predict([x_test])
plt.imshow(predictions[0,:,:,0])
plt.show()

#Now, use the testGenerator function as you use it in the trainUnet notebook
#and play with the scale value to see if you see something
testGene = testGenerator("/test", 5)
img = next(testGene)
scale = 200
plt.subplot(1, 2, 1)
plt.imshow(im[0,:,:,0] * scale)
plt.subplot(1, 2, 2)
predictions = model.predict([im])
plt.imshow(predictions[0,:,:,0])
plt.show()

Once you determined that the problem is in testGene go and modify it as follow:

def testGenerator(test_path,num_image = 30,target_size = (256,256),flag_multi_class = False,as_gray = True):
    for i in range(num_image):
        img = io.imread(os.path.join(test_path,"%d.png"%i),as_gray = as_gray)
        #img=img/255.
        #img = trans.resize(img,target_size)
        img = np.reshape(img,img.shape+(1,)) if (not flag_multi_class) else img
        img = np.reshape(img,(1,)+img.shape)
        yield img

And to complete the modification I also use some code from other answer to get binary masks:

def saveResult(save_path,npyfile,flag_multi_class = False,num_class = 2):
    for i,item in enumerate(npyfile):
        if flag_multi_class:
            img = labelVisualize(num_class,COLOR_DICT,item)
        else:
            img=item[:,:,0]
            print(np.max(img),np.min(img))
            img[img>0.5]=1
            img[img<=0.5]=0
            print(np.max(img),np.min(img))
            io.imsave(os.path.join(save_path,"%d_predict.png"%i),img)

[Antonio-Nappi]

@jacobwhgordon even the up conv2d needs the BatchNormalization?

[jacobwhgordon]

@Antonio-Nappi No, I did not add the batch normalizations to the up# layers, just the conv# layers (I realize this response is 2 years late, but maybe it will help someone in the future) so, for example, the 6th block looked like this for me:

    up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))
    merge6 = concatenate([drop4,up6], axis = 3)
    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)
    conv6 = BatchNormalization()(conv6)
    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)
    conv6 = BatchNormalization()(conv6)

@Thunder003 The conv10 layer doesn't follow the same convention as all the other blocks and in the 2dconv there is a sigmoid instead of a ReLU. I believe the conv10 layer should just be there to turn our outputs into 1s and 0s. Putting a batch normalization layer there would be normalizing our output pixel classes, which probably is not the desired outcome?

[marlon-shiftone]

Adding batch nornalization layers has fixed my problem. Thanks for the tips!

[dayangkunurfaizah]

Hello there. Sorry, I hope that my question is not too late for you guys to answer me. I also have the same problem in which the predicted image become a black/white image. I use to add batchnormalization layer every layer, but won't solve the problem. `So here's my code:-

`import os import glob os.environ["CUDA_VISIBLE_DEVICES"] = "0" import numpy as np from keras.models import from keras.layers import Input, merge, Conv2D, MaxPooling2D, UpSampling2D, Dropout, Cropping2D, Concatenate, BatchNormalization from keras.optimizers import from keras.callbacks import ModelCheckpoint, LearningRateScheduler from keras import backend as keras from data import *

class myUnet(object): def init(self, img_rows = 512, img_cols = 512): self.img_rows = img_rows self.img_cols = img_cols

def load_data(self):
    mydata = dataProcess(self.img_rows, self.img_cols)
    imgs_train, imgs_mask_train = mydata.load_train_data()
    imgs_test = mydata.load_test_data()
    return imgs_train, imgs_mask_train, imgs_test

def get_unet(self):
    inputs = Input((self.img_rows, self.img_cols,1))

    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)
    conv1 = BatchNormalization()(conv1)
    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)
    conv1 = BatchNormalization()(conv1)
    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)

    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)
    conv2 = BatchNormalization()(conv2)
    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)
    conv2 = BatchNormalization()(conv2)
    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)

    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2)
    conv3 = BatchNormalization()(conv3)
    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3)
    conv3 = BatchNormalization()(conv3)
    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)

    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3)
    conv4 = BatchNormalization()(conv4)
    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4)
    conv4 = BatchNormalization()(conv4)
    drop4 = Dropout(0.5)(conv4)
    pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)

    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)
    conv5 = BatchNormalization()(conv5)
    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)
    conv5 = BatchNormalization()(conv5)
    drop5 = Dropout(0.5)(conv5)

    up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))
    merge6 = Concatenate(axis=3)([drop4, up6])
    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)
    conv6 = BatchNormalization()(conv6)
    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)
    conv6 = BatchNormalization()(conv6)

    up7 = Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv6))
    merge7 = Concatenate(axis=3)([conv3, up7])
    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)
    conv7 = BatchNormalization()(conv7)
    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)
    conv7 = BatchNormalization()(conv7)

    up8 = Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv7))
    merge8 = Concatenate(axis=3)([conv2,up8])
    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)
    conv8 = BatchNormalization()(conv8)
    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)
    conv8 = BatchNormalization()(conv8)

    up9 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv8))
    merge9 = Concatenate(axis=3)([conv1,up9])
    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)
    conv9 = BatchNormalization()(conv9)
    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)
    conv9 = BatchNormalization()(conv9)
    conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)

    conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)

    model = Model(input = inputs, output = conv10)
    model.compile(optimizer = Adam(lr = 1e-4), loss = 'binary_crossentropy', metrics = ['accuracy'])
    return model

def train(self):
    print("loading data")
    imgs_train, imgs_mask_train, imgs_test = self.load_data()
    print("loading data done")
    model = self.get_unet()
    print("got unet")
    model_checkpoint = ModelCheckpoint('my_unet.hdf5', monitor='loss',verbose=1, save_best_only=True)
    print('Fitting model...')
    model.fit(imgs_train, imgs_mask_train, batch_size=2, epochs=10, verbose=1,validation_split=0.2, shuffle=True, callbacks=[model_checkpoint])
    print('predict test data')
    imgs_mask_test = model.predict(imgs_test, batch_size=1, verbose=1)
    np.save('/home/motmot/Documents/unet7/results/imgs_mask_test.npy', imgs_mask_test)

def save_img(self):
    print("array to image")
    imgs = np.load('/home/motmot/Documents/unet7/results/imgs_mask_test.npy')
    imgs_name = sorted(glob.glob("/home/motmot/Documents/unet7/raw/test"+"/*."+"jpg"))
    for i in range(imgs.shape[0]):
        img = imgs[i]
        imgname = imgs_name[i]
        midname = imgname[imgname.rindex("/") + 1:]
        img_order = midname[:-4]
        img = array_to_img(img)
        img.save("/home/motmot/Documents/unet7/results/%s.jpg"%(img_order))

if name == 'main': myunet = myUnet() myunet.train() myunet.save_img()

And I got my predict.py error. The error says here:-

ValueError Traceback (most recent call last)

in () 9 myunet = myUnet() 10 model = myunet.get_unet() ---> 11 model.load_weights('my_unet.hdf5') 12 imgs_mask_test = model.predict(imgs_test, batch_size=2, verbose=1) 13 print("save imgs_mask_test.npy") /home/motmot/anaconda3/envs/py35/lib/python3.5/site-packages/keras/engine/network.py in load_weights(self, filepath, by_name, skip_mismatch, reshape) 1159 else: 1160 saving.load_weights_from_hdf5_group( -> 1161 f, self.layers, reshape=reshape) 1162 1163 def _updated_config(self): /home/motmot/anaconda3/envs/py35/lib/python3.5/site-packages/keras/engine/saving.py in load_weights_from_hdf5_group(f, layers, reshape) 898 'containing ' + str(len(layer_names)) + 899 ' layers into a model with ' + --> 900 str(len(filtered_layers)) + ' layers.') 901 902 # We batch weight value assignments in a single backend call ValueError: You are trying to load a weight file containing 43 layers into a model with 24 layers.`