How to run with another dataset and want to get the requirement for your project

An-BN commented 3 years ago

Hi, I‘m interested in your project and ET-Net, and I'm trying to apply ET-net to the dataset refuge. So can you public the requirement to run your project and tell me how to run it with another dataset like refuge. Thank you so much.

ZM-J commented 3 years ago

Sorry, since the project is still in the process of development, I forget to generate a requirement.txt. Basically, you just need latest version of popular libraries like PyTorch, PIL, NumPy, matplotlib, sklearn, etc. To run it with another dataset:

Write dataset classes (XXDataset class to generate (image, label, edge), XXTestDataset class to generate (image, mask), XXMetricDataset class to generate (image, label, edge, mask)) and put it in datasets folder
Add your dataset to utils/get_dataset.py and construct your dataset
Change dataset in args.py
Follow instructions in README.md

It's my pleasure to implement this task whose code hasn't been released officially yet.

An-BN commented 3 years ago

It would be my honor to get your help. Could you show me the structure of the dataset DRIVE or CHASE-DB1 in your directories. Thanks

ZM-J commented 3 years ago

I never expected that this repo would be interested by other one so quickly, and I would release the structure of these directories in this repo before long. You can define your own datasets based on your dataset class implementation. It is recommended like this:

+---CHASEDB1
|   +---train
|   |   +---1stHO
|   |   |       Image_01L_1stHO.png
|   |   |       Image_01R_1stHO.png
|   |   |       Image_02L_1stHO.png
|   |   |       Image_02R_1stHO.png
|   |   |       Image_03L_1stHO.png
|   |   |       Image_03R_1stHO.png
|   |   |       Image_04L_1stHO.png
|   |   |       Image_04R_1stHO.png
|   |   |       Image_05L_1stHO.png
|   |   |       Image_05R_1stHO.png
|   |   |       Image_06L_1stHO.png
|   |   |       Image_06R_1stHO.png
|   |   |       Image_07L_1stHO.png
|   |   |       Image_07R_1stHO.png
|   |   |       Image_08L_1stHO.png
|   |   |       Image_08R_1stHO.png
|   |   |       Image_09L_1stHO.png
|   |   |       Image_09R_1stHO.png
|   |   |       Image_10L_1stHO.png
|   |   |       Image_10R_1stHO.png
|   |   |       
|   |   +---2ndHO
|   |   |       Image_01L_2ndHO.png
|   |   |       Image_01R_2ndHO.png
|   |   |       Image_02L_2ndHO.png
|   |   |       Image_02R_2ndHO.png
|   |   |       Image_03L_2ndHO.png
|   |   |       Image_03R_2ndHO.png
|   |   |       Image_04L_2ndHO.png
|   |   |       Image_04R_2ndHO.png
|   |   |       Image_05L_2ndHO.png
|   |   |       Image_05R_2ndHO.png
|   |   |       Image_06L_2ndHO.png
|   |   |       Image_06R_2ndHO.png
|   |   |       Image_07L_2ndHO.png
|   |   |       Image_07R_2ndHO.png
|   |   |       Image_08L_2ndHO.png
|   |   |       Image_08R_2ndHO.png
|   |   |       Image_09L_2ndHO.png
|   |   |       Image_09R_2ndHO.png
|   |   |       Image_10L_2ndHO.png
|   |   |       Image_10R_2ndHO.png
|   |   |       
|   |   +---edge
|   |   |       Image_01L_1stHO.png
|   |   |       Image_01R_1stHO.png
|   |   |       Image_02L_1stHO.png
|   |   |       Image_02R_1stHO.png
|   |   |       Image_03L_1stHO.png
|   |   |       Image_03R_1stHO.png
|   |   |       Image_04L_1stHO.png
|   |   |       Image_04R_1stHO.png
|   |   |       Image_05L_1stHO.png
|   |   |       Image_05R_1stHO.png
|   |   |       Image_06L_1stHO.png
|   |   |       Image_06R_1stHO.png
|   |   |       Image_07L_1stHO.png
|   |   |       Image_07R_1stHO.png
|   |   |       Image_08L_1stHO.png
|   |   |       Image_08R_1stHO.png
|   |   |       Image_09L_1stHO.png
|   |   |       Image_09R_1stHO.png
|   |   |       Image_10L_1stHO.png
|   |   |       Image_10R_1stHO.png
|   |   |       
|   |   \---images
|   |           Image_01L.jpg
|   |           Image_01R.jpg
|   |           Image_02L.jpg
|   |           Image_02R.jpg
|   |           Image_03L.jpg
|   |           Image_03R.jpg
|   |           Image_04L.jpg
|   |           Image_04R.jpg
|   |           Image_05L.jpg
|   |           Image_05R.jpg
|   |           Image_06L.jpg
|   |           Image_06R.jpg
|   |           Image_07L.jpg
|   |           Image_07R.jpg
|   |           Image_08L.jpg
|   |           Image_08R.jpg
|   |           Image_09L.jpg
|   |           Image_09R.jpg
|   |           Image_10L.jpg
|   |           Image_10R.jpg
|   |           
|   \---val
|       +---1stHO
|       |       Image_11L_1stHO.png
|       |       Image_11R_1stHO.png
|       |       Image_12L_1stHO.png
|       |       Image_12R_1stHO.png
|       |       Image_13L_1stHO.png
|       |       Image_13R_1stHO.png
|       |       Image_14L_1stHO.png
|       |       Image_14R_1stHO.png
|       |       
|       +---2ndHO
|       |       Image_11L_2ndHO.png
|       |       Image_11R_2ndHO.png
|       |       Image_12L_2ndHO.png
|       |       Image_12R_2ndHO.png
|       |       Image_13L_2ndHO.png
|       |       Image_13R_2ndHO.png
|       |       Image_14L_2ndHO.png
|       |       Image_14R_2ndHO.png
|       |       
|       +---edge
|       |       Image_11L_1stHO.png
|       |       Image_11R_1stHO.png
|       |       Image_12L_1stHO.png
|       |       Image_12R_1stHO.png
|       |       Image_13L_1stHO.png
|       |       Image_13R_1stHO.png
|       |       Image_14L_1stHO.png
|       |       Image_14R_1stHO.png
|       |       
|       \---images
|               Image_11L.jpg
|               Image_11R.jpg
|               Image_12L.jpg
|               Image_12R.jpg
|               Image_13L.jpg
|               Image_13R.jpg
|               Image_14L.jpg
|               Image_14R.jpg
|               
\---DRIVE
    +---test
    |   +---images
    |   |       01_test.tif
    |   |       02_test.tif
    |   |       03_test.tif
    |   |       04_test.tif
    |   |       05_test.tif
    |   |       06_test.tif
    |   |       07_test.tif
    |   |       08_test.tif
    |   |       09_test.tif
    |   |       10_test.tif
    |   |       11_test.tif
    |   |       12_test.tif
    |   |       13_test.tif
    |   |       14_test.tif
    |   |       15_test.tif
    |   |       16_test.tif
    |   |       17_test.tif
    |   |       18_test.tif
    |   |       19_test.tif
    |   |       20_test.tif
    |   |       
    |   \---mask
    |           01_test_mask.gif
    |           02_test_mask.gif
    |           03_test_mask.gif
    |           04_test_mask.gif
    |           05_test_mask.gif
    |           06_test_mask.gif
    |           07_test_mask.gif
    |           08_test_mask.gif
    |           09_test_mask.gif
    |           10_test_mask.gif
    |           11_test_mask.gif
    |           12_test_mask.gif
    |           13_test_mask.gif
    |           14_test_mask.gif
    |           15_test_mask.gif
    |           16_test_mask.gif
    |           17_test_mask.gif
    |           18_test_mask.gif
    |           19_test_mask.gif
    |           20_test_mask.gif
    |           
    +---training
    |   +---1st_manual
    |   |       21_manual1.gif
    |   |       22_manual1.gif
    |   |       23_manual1.gif
    |   |       24_manual1.gif
    |   |       25_manual1.gif
    |   |       26_manual1.gif
    |   |       27_manual1.gif
    |   |       28_manual1.gif
    |   |       29_manual1.gif
    |   |       30_manual1.gif
    |   |       31_manual1.gif
    |   |       32_manual1.gif
    |   |       33_manual1.gif
    |   |       34_manual1.gif
    |   |       35_manual1.gif
    |   |       36_manual1.gif
    |   |       
    |   +---edge
    |   |       21_manual1.gif
    |   |       22_manual1.gif
    |   |       23_manual1.gif
    |   |       24_manual1.gif
    |   |       25_manual1.gif
    |   |       26_manual1.gif
    |   |       27_manual1.gif
    |   |       28_manual1.gif
    |   |       29_manual1.gif
    |   |       30_manual1.gif
    |   |       31_manual1.gif
    |   |       32_manual1.gif
    |   |       33_manual1.gif
    |   |       34_manual1.gif
    |   |       35_manual1.gif
    |   |       36_manual1.gif
    |   |       another_21.gif
    |   |       
    |   +---images
    |   |       21_training.tif
    |   |       22_training.tif
    |   |       23_training.tif
    |   |       24_training.tif
    |   |       25_training.tif
    |   |       26_training.tif
    |   |       27_training.tif
    |   |       28_training.tif
    |   |       29_training.tif
    |   |       30_training.tif
    |   |       31_training.tif
    |   |       32_training.tif
    |   |       33_training.tif
    |   |       34_training.tif
    |   |       35_training.tif
    |   |       36_training.tif
    |   |       
    |   \---mask
    |           21_training_mask.gif
    |           22_training_mask.gif
    |           23_training_mask.gif
    |           24_training_mask.gif
    |           25_training_mask.gif
    |           26_training_mask.gif
    |           27_training_mask.gif
    |           28_training_mask.gif
    |           29_training_mask.gif
    |           30_training_mask.gif
    |           31_training_mask.gif
    |           32_training_mask.gif
    |           33_training_mask.gif
    |           34_training_mask.gif
    |           35_training_mask.gif
    |           36_training_mask.gif
    |           
    \---val
        |   edge.zip
        |   
        +---1st_manual
        |       37_manual1.gif
        |       38_manual1.gif
        |       39_manual1.gif
        |       40_manual1.gif
        |       
        +---edge
        |       37_manual1.gif
        |       38_manual1.gif
        |       39_manual1.gif
        |       40_manual1.gif
        |       
        +---images
        |       37_training.tif
        |       38_training.tif
        |       39_training.tif
        |       40_training.tif
        |       
        \---mask
                37_training_mask.gif
                38_training_mask.gif
                39_training_mask.gif
                40_training_mask.gif

An-BN commented 3 years ago

ndrishtiGS_089 Hi, I still trying run ET-Net with REFUGE dataset, the label is some thing like that, and output of segment in this dataset have 3 class for every pixel. I changed ARGS[egm] = [3, 64] and ARGS[wam] = [64, 3] and something when render a result images but some thing still go wrong. I thing the problem is the edge of input, a total must have 3 regions to segment. Any suggest and help me to run in this dataset. It very important to me so i hope to see your reply. Thank you!

ZM-J commented 3 years ago

Hi, I still trying run ET-Net with REFUGE dataset, the label is some thing like that, and output of segment in this dataset have 3 class for every pixel. I changed ARGS[egm] = [3, 64] and ARGS[wam] = [64, 3] and something when render a result images but some thing still go wrong. I thing the problem is the edge of input, a total must have 3 regions to segment. Any suggest and help me to run in this dataset. It very important to me so i hope to see your reply. Thank you!

I think first you may not know how to get the edges of the picture described above. I think the simplest way to solve it is:

Make the gray part to be black, and run utils/get_edges.py to get edge 1
Make the gray part to be white, and run utils/get_edges.py to get edge 2
Merge edges acquired above

And second, you may have to change all metrics for binary classification, e.g., IoU, to 3-class classification ones.

An-BN commented 3 years ago

Hi, I did as you say and get the edges of the picture.

I trained and loss, IoU on train and val look good, but when i predict, although output_channels = 3 but only 2 classes have pixels. This my results:

Any help about this. Thank you

An-BN commented 3 years ago

I try print number of pixel have probability in class 2 != 0, here my code and output:

import numpy as np 
import cv2
from PIL import Image

out_net = np.load('full_test.npy')
print(f"original image size: {out_net.shape}")

probability_twos_nonezero = (out_net[:, 2, :, :] != 0).sum()
print(f"Num pixel of class 2 have probability != 0: {probability_twos_nonezero}")

pred = np.zeros((out_net.shape[2], out_net.shape[3]))
for i in range(out_net.shape[2]):
  for j in range(out_net.shape[3]):
    if np.argmax(out_net[:, :, i, j]) == 0:
      pred[i, j] = 0
    elif np.argmax(out_net[:, :, i, j]) == 1:
      pred[i, j] = 128
    else:
      pred[i, j] = 255
num_zeros = (pred == 0).sum()
num_ones = (pred == 128).sum()
num_twos = (pred == 255).sum()
print(f"Num pixel of class 0: {num_zeros}")
print(f"Num pixel of class 1: {num_ones}")
print(f"Num pixel of class 2: {num_twos}")

ZM-J commented 3 years ago

You may need to change test.py as well, like you should make test results corresponding to 3 classes, but not a single probability map for 2-class segmentation. Like, in test.py line 55, it shouldn't be '''python3 test_results = test_results[:, 1, :images.size(2), :images.size(3)] * mask ''' You may need to keep each channel as their class, e.g., test_results[:, 0, :images.size(2), :images.size(3)] is the probability map of class 0 (background), test_results[:, 1, :images.size(2), :images.size(3)] is class 1, test_results[:, 2, :images.size(2), :images.size(3)] is class 2

An-BN commented 3 years ago

Hi, i think the problem is in class NAMEDataset():

label = label.convert('1')
edge = edge.convert('1')        
mask = mask.convert('1')

img = np.array(img).transpose((2, 0, 1)) / 255. # [0, 255] -> [0, 1]
mean = np.array([123.68, 116.779, 103.939]).reshape((3, 1, 1)) / 255
img = torch.FloatTensor(img - mean)

label = torch.LongTensor(np.array(label)) // 255 # 512 x 512 # [0, 255] -> [0, 1]
edge = torch.LongTensor(np.array(edge)) // 255 # 512 x 512
mask = torch.LongTensor(np.array(mask)) // 255 # 512 x 512
return {'image': img, 'label': label, 'edge': edge, 'mask': mask}

It maybe using one-hot vector like [1, 0, 0], [0, 0, 1], [0, 1, 0] to present for 3 classes in label, edge. Anyway, i want to ask how do you get the mean value mean = np.array([123.68, 116.779, 103.939]).reshape((3, 1, 1)) / 255 And with 3 classes what i must change in code in the above. Thank you!

ZM-J commented 3 years ago

Hi, i think the problem is in class NAMEDataset():
label = label.convert('1')
edge = edge.convert('1')        
mask = mask.convert('1')

img = np.array(img).transpose((2, 0, 1)) / 255. # [0, 255] -> [0, 1]
mean = np.array([123.68, 116.779, 103.939]).reshape((3, 1, 1)) / 255
img = torch.FloatTensor(img - mean)

label = torch.LongTensor(np.array(label)) // 255 # 512 x 512 # [0, 255] -> [0, 1]
edge = torch.LongTensor(np.array(edge)) // 255 # 512 x 512
mask = torch.LongTensor(np.array(mask)) // 255 # 512 x 512
return {'image': img, 'label': label, 'edge': edge, 'mask': mask}
It maybe using one-hot vector like [1, 0, 0], [0, 0, 1], [0, 1, 0] to present for 3 classes in label, edge. Anyway, i want to ask how do you get the mean value mean = np.array([123.68, 116.779, 103.939]).reshape((3, 1, 1)) / 255 And with 3 classes what i must change in code in the above. Thank you!

I am very appreciated that this code works for you.

Moreover, this mean is from ImageNet because ResNet (encoder) asks you to do so.

An-BN commented 3 years ago

You may need to change test.py as well, like you should make test results corresponding to 3 classes, but not a single probability map for 2-class segmentation. Like, in test.py line 55, it shouldn't be '''python3 test_results = test_results[:, 1, :images.size(2), :images.size(3)] * mask ''' You may need to keep each channel as their class, e.g., test_results[:, 0, :images.size(2), :images.size(3)] is the probability map of class 0 (background), test_results[:, 1, :images.size(2), :images.size(3)] is class 1, test_results[:, 2, :images.size(2), :images.size(3)] is class 2

Thank for the reply. At reply before, I know that so when run test, i saved all test_results[:, :, :images.size(2), :images.size(3)] and when save final image i using 2 loop in range(images.size(2)) and range(images.size(3)) to calculate the class in the whole test_results but some thing still wrong! Any idea to help me? Please!!!!

An-BN commented 3 years ago

@ZM-J Hi, about this code

label = label.convert('1')
edge = edge.convert('1')        
mask = mask.convert('1')

img = np.array(img).transpose((2, 0, 1)) / 255. # [0, 255] -> [0, 1]
mean = np.array([123.68, 116.779, 103.939]).reshape((3, 1, 1)) / 255
img = torch.FloatTensor(img - mean)

label = torch.LongTensor(np.array(label)) // 255 # 512 x 512 # [0, 255] -> [0, 1]
edge = torch.LongTensor(np.array(edge)) // 255 # 512 x 512
mask = torch.LongTensor(np.array(mask)) // 255 # 512 x 512
return {'image': img, 'label': label, 'edge': edge, 'mask': mask}

In my label and edge, 3 value is (0, 128, 255) so do you think it could be //127 to get class can be (0, 1, 2) like:

label = label.convert('1')
edge = edge.convert('1')        
mask = mask.convert('1')

img = np.array(img).transpose((2, 0, 1)) / 255. # [0, 255] -> [0, 1]
mean = np.array([123.68, 116.779, 103.939]).reshape((3, 1, 1)) / 255
img = torch.FloatTensor(img - mean)

label = torch.LongTensor(np.array(label)) // 127 
edge = torch.LongTensor(np.array(edge)) // 127

return {'image': img, 'label': label, 'edge': edge, 'mask': mask}

ZM-J commented 3 years ago

I think it is okay. Have you tried it?

An-BN commented 3 years ago

I have already tried it, but in this case, the number of pixel in class 1 = 0

ZM-J commented 3 years ago

So in your previous issues, # pixel of class 2 = 0, but now , this becomes # pixel of class 1 = 0.

Why? I have no idea about it😟

An-BN commented 3 years ago

I solved the problem. The issues in DATASETName_dataset.py

label = label.convert('1')
edge = edge.convert('1')

because convert('1') so when i labeling for the edge and label, i missing my class. that would be:

label = label.convert('L')
edge = edge.convert('L')

and:

label[label == 0] = 0
label[label == 128] = 1
label[label == 255] = 2

Thank you just the same

ZM-J commented 3 years ago

It's my pleasure to help you run the code. Plz show me some of your results (quantative & qualitative) if you've successed running this program.

An-BN commented 3 years ago

This is the result for refuge dataset before and after i smoothing the edge,

Dice Optic Cup = 0.8239790403998666
Dice Optic Disc = 0.9323068727563768
MAE CDR = 0.06401355197352569

Dice Optic Cup = 0.8276616853490331
Dice Optic Disc = 0.933879655017118
MAE CDR = 0.05251367493878562

It is lower than the paper ET-Net reported I first localize the disc centers following the existing automatic disc detection method as in paper said too. Everything is the same in paper excepted 128128 patches. I am trying to not using that, can u tell me what must be change if i not use 128 128 patches

ZM-J commented 3 years ago

You may change everything using _random_crop to another one, say, _resize to a square that is near original size, like, 512x512. In testing procedure, you may the prediction process, say, first _resize to 512x512, then predict the probability map, and last interpolate the probability map to the original size of the picture. BTW, could you show us some prediction results? We are glad to see it! And if you think that there is no further issue, please consider closing this issue😀

An-BN commented 3 years ago

This some prediction results:

and label corresponding:

An-BN commented 3 years ago

You may change everything using _random_crop to another one, say, _resize to a square that is near original size, like, 512x512. In testing procedure, you may the prediction process, say, first _resize to 512x512, then predict the probability map, and last interpolate the probability map to the original size of the picture. BTW, could you show us some prediction results? We are glad to see it! And if you think that there is no further issue, please consider closing this issue😀

I closing this issue here. Thank you again, your code is great and i learn a lot from you.

ZM-J / ET-Net

How to run with another dataset and want to get the requirement for your project #2