How to run???

[kimsangseob]

Hello, I'm a student and I'm just started about DL. I want to use this code but I don't know how to start.

Can you help me...? Thanks.

[sainatarajan]

@kimsangseob Glad to help, but have you gone through the readme? If yes, and you're still unable to follow it, let me know the problem you're facing.

[kimsangseob]

I want run data.py in cmd. Then I put python data.py it gonna be works!!

But I face another problem.. Its like this. Do you know why??

[sainatarajan]

I think there seems to be a problem with the value of your labelpath variable. Can you print the value of the labelpath variable before the line label= load_img(labelpath, grayscale= True, target_size= [512, 512]) and tell me?

[kimsangseob]

You mean in data.py?

[kimsangseob]

Here is data.py!!

[kimsangseob]

Use : Windows 10. Tensorflow==1.5.0, Keras==2.0.8, Python==3.6.10

Data set : png images(I modified code jpg to png)

Lable : png file(labeled with photoshop with red line(255, 0, 0) and then change it binary mask with ImageJ

[sainatarajan]

I meant in data.py, add this line: print(labelpath, pic_name) before the line 31 label = load_img(labelpath, grayscale=True, target_size=[512, 512]) and then run your code and then, can you show me the output?

My expected guess is that:

self.label_path is assigned the value: self.label_path= "C:\Users\user\Desktop\U-Net-master\data\train\labels"

When the program executes the line 31 i.e. labelpath= self.label_path + '/' + pic_name

labelpath is assigned the value "C:\Users\user\Desktop\U-Net-master\data\train\labels" + '/' + "C:\Users\user\Desktop\U-Net-master\data\train\labels\0.png" which is wrong. There's no path that exists like that and hence the error. What you need in order to solve the error is that pic_name must have the value 0.png instead of the whole path to the file.

So edit the line 31 to:

labelpath= os.path.join(self.label_path, os.path.basename(pic_name))

This should solve the error.

[kimsangseob]

I'm sorry I got this error...

[sainatarajan]

@kimsangseob You forgot to include the closing parenthesis ) in line 11.

[kimsangseob]

I fixed it and then got this error.

[sainatarajan]

There's no variable called filename and hence the error. Since you're not using the variable filename anywhere else, if you could remove or comment that line, it should run.

[kimsangseob]

I removed that line and got this error again...T.T

[sainatarajan]

I meant in data.py, add this line: print(labelpath, pic_name) before the line 31 label = load_img(labelpath, grayscale=True, target_size=[512, 512]) and then run your code and then, can you show me the output?

My expected guess is that:

self.label_path is assigned the value: self.label_path= "C:\Users\user\Desktop\U-Net-master\data\train\labels"

When the program executes the line 31 i.e. labelpath= self.label_path + '/' + pic_name

labelpath is assigned the value "C:\Users\user\Desktop\U-Net-master\data\train\labels" + '/' + "C:\Users\user\Desktop\U-Net-master\data\train\labels\0.png" which is wrong. There's no path that exists like that and hence the error. What you need in order to solve the error is that pic_name must have the value 0.png instead of the whole path to the file.

So edit the line 31 to:

labelpath= os.path.join(self.label_path, os.path.basename(pic_name))

This should solve the error.

This is the solution. Can you try this?

[kimsangseob]

I added print(labelpath, pic_name) already. Like this!

from keras.preprocessing.image import img_to_array, load_img import numpy as np import glob

class dataProcess(object): def init(self, out_rows, out_cols, data_path=r"C:\Users\user\Desktop\U-Net-master\data\train\images", label_path=r"C:\Users\user\Desktop\U-Net-master\data\train\labels", test_path=r"C:\Users\user\Desktop\U-Net-master\data\test\images", npy_path=r"C:\Users\user\Desktop\U-Net-master\data\npydata", img_type="png"): self.out_rows = out_rows self.out_cols = out_cols self.data_path = data_path self.label_path = label_path self.img_type = img_type self.test_path = test_path self.npy_path = npy_path

def create_train_data(self):
    i = 0
    print('Creating training images...')
    imgs = glob.glob(self.data_path+"/*."+self.img_type)
    print(len(imgs))
    imgdatas = np.ndarray((len(imgs), self.out_rows, self.out_cols, 3), dtype=np.uint8)
    imglabels = np.ndarray((len(imgs), self.out_rows, self.out_cols, 1), dtype=np.uint8)

    for x in range(len(imgs)):
        imgpath = imgs[x]
        pic_name = imgpath.split('/')[-1]
        labelpath = self.label_path + '/' + pic_name
        img = load_img(imgpath, grayscale=False, target_size=[512, 512])
        label = load_img(labelpath, grayscale=True, target_size=[512, 512])
        print(labelpath, pic_name)
        img = img_to_array(img)
        label = img_to_array(label)
        imgdatas[i] = img
        imglabels[i] = label
        if i % 100 == 0:
            print('Done:', len(imgs),' images')
        i += 1

    print('loading done')
    np.save(self.npy_path + '/imgs_train.npy', imgdatas)
    np.save(self.npy_path + '/imgs_mask_train.npy', imglabels)
    print('Saving to .npy files done.')

def create_test_data(self):
    i = 0
    print('Creating test images...')
    imgs = glob.glob(self.test_path + "/*." + self.img_type)
    imgdatas = np.ndarray((len(imgs), self.out_rows, self.out_cols, 3), dtype=np.uint8)
    testpathlist = []

    for imgname in imgs:
        testpath = imgname
        testpathlist.append(testpath)
        img = load_img(testpath, grayscale=False, target_size=[512, 512])
        img = img_to_array(img)
        imgdatas[i] = img
        i += 1

    txtname = './data/results/pic.txt'
    with open(txtname, 'w') as f:
        for i in range(len(testpathlist)):
            f.writelines(testpathlist[i] + '\n')
    print('loading done')
    np.save(self.npy_path + '/imgs_test.npy', imgdatas)
    print('Saving to imgs_test.npy files done.')

def load_train_data(self):
    print('load train images...')
    imgs_train = np.load(self.npy_path + "/imgs_train.npy")
    imgs_mask_train = np.load(self.npy_path + "/imgs_mask_train.npy")
    imgs_train = imgs_train.astype('float32')
    imgs_mask_train = imgs_mask_train.astype('float32')
    imgs_train /= 255
    imgs_mask_train /= 255
    imgs_mask_train[imgs_mask_train > 0.5] = 1  
    imgs_mask_train[imgs_mask_train <= 0.5] = 0 
    return imgs_train, imgs_mask_train

def load_test_data(self):
    print('-' * 30)
    print('load test images...')
    print('-' * 30)
    imgs_test = np.load(self.npy_path + "/imgs_test.npy")
    imgs_test = imgs_test.astype('float32')
    imgs_test /= 255
    return imgs_test

if name == "main": mydata = dataProcess(512, 512) mydata.create_train_data() mydata.create_test_data()

[sainatarajan]

You added the print line at the wrong place. Can you add the print line before the line: label = load_img(labelpath, grayscale=True, target_size=[512, 512])

[kimsangseob]

I got this again.. I'm really sorry about I think now bothering you.

[kimsangseob]

And I changed it! Really thank you.

[sainatarajan]

I got this again.. I'm really sorry about I think now bothering you.

It's okay. No worries. As I said in my earlier comment, you need to change the line from this: labelpath = self.label_path + '/' + pic_name to this: labelpath= os.path.join(self.label_path, os.path.basename(pic_name))

[kimsangseob]

Error is like this and I modified code.

[sainatarajan]

Can you add the line import os at the top of the file?

[kimsangseob]

Maybe you solved my problem!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Thanks I'm really happy T.T

[kimsangseob]

Oops... I tried next step and I got this..

[sainatarajan]

This seems to be a TensorFlow/Keras issue. I had used the tensorflow version 1.14 and keras version 2.2.4 when coding this repo. Could you also try installing these two specific versions from the below commands one after the other and then try it out? pip install tensorflow==1.14 pip install keras==2.2.4

[kimsangseob]

I changed version of tensorflow and keras. Then I got ModuleNotFoundError : No module named 'cv2'

[sainatarajan]

Interesting. How come installing tf and keras, modifies opencv! Can you install opencv using this command: pip install opencv-python and then run the code?

[kimsangseob]

Oh!!!!!! maybe it works@!!@!@!@ Thanks a lot!!!! Actually I don't know about opencv specifically... This is very first time to use DeepLearning.

I never person who really help me a lot.. I will cry because of your cherish ㅠㅠ Thanks...

[sainatarajan]

You're welcome! Feel free to ask further doubts in case any issue arises.

[kimsangseob]

You are a my life saver!! :)

[kimsangseob]

Oh If I want to use gpu then : tensorflow-gpu==1.14 keras==2.2.4 python==3.6.10

and how about cuda and cudnn??

[sainatarajan]

Yes, pip install tensorflow-gpu==1.14 would be enough. You don't need to install python/keras. TensorflowGPU comes along with CuDNN and cuda.

[sainatarajan]

However, I suggest, you learn PyTorch instead of Tensorflow/Keras.

[kimsangseob]

Thanks for your advice!!

Now I tried using gpu for train but I got this.

[kimsangseob]

Maybe I can solve it to just install cuda 10.0 right??

[sainatarajan]

You're right. I think your system has CUDA version < 10.0 installed.

[kimsangseob]

Okay! I'll try!! THANKS!!

[kimsangseob]

Excuse me. I think I finished learning then how do you think about it?

[sainatarajan]

I think the model has stopped training abruptly. The error: Resource Exhausted: OOM when allocating tensor indicates that the GPU was low on memory and therefore the requested memory space was not allocated.

What you could do is to reduce the batch_size. At present, it's set to 4, but you could reduce it to 2 in the line model.fit() in the unet.py file. Which GPU are you using?

[kimsangseob]

I have like this!

[kimsangseob]

I modified it and then it seems like works!

[sainatarajan]

I have like this!

I see. Your current setup should easily be able to work with large batch sizes.

[kimsangseob]

But it looks like using just 1 gpu aroun 10% and another 7% and other 2 are note using.

I changed batch size 4 to 2.

[sainatarajan]

import tensorflow as tf

config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth = True
tf.compat.v1.keras.backend.set_session(tf.compat.v1.Session(config=config))

In case if the error arises in the future, could you add these lines in the unet.py file after the from data import * line?

[kimsangseob]

I add that lines and then get this error.

[kimsangseob]

I changed like this!!

[sainatarajan]

@kimsangseob I think it's the same issue that occurred with the labelpath variable. Add this line: print(path) before the line img.save(path) and then show me the output. Or edit this line from: path = "./data/results/" + piclist[i] to: path = "./data/results/" + os.path.basename(piclist[i])

If I'm correct, this should work.

[kimsangseob]

First, I tried add print(path) before the line img.save(path) and this is results!

Now I'll try netx one. So you mean when I try second one I have to remove print(path) whichone I added it, and then change path = "~" + ~ right???

[sainatarajan]

Your print statement can be there. It won't cause any issue. I can see that the error arose because of the value of the path variable.

[kimsangseob]

Okay then left print(path) and edit that one.

[kimsangseob]

Maybe it works@!@#@#@#!#@#!@!!@!~@@~@#@#!#

Right???

[sainatarajan]

Yup. It's working!