U-Net 2D ValueError when preparing data for training

[shoyip]

Describe the bug I am currently using the ZeroCostDL4Mic U-Net 2D notebook. Everything is smooth until I get to step 4.1. Running the cell causes the following ValueError

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

[<ipython-input-11-d5e38c518559>](https://localhost:8080/#) in <cell line: 15>()
     13 
     14 print('Getting class weights...')
---> 15 class_weights = getClassWeights(Training_target)
     16 
     17 # --------------------- Using pretrained model ------------------------

[<ipython-input-2-7bc0f39ed880>](https://localhost:8080/#) in getClassWeights(Training_target_path)
    237     mask = io.imread(os.path.join(Training_target_path, Mask_dir_list[i]))
    238     mask = normalizeMinMax(mask)
--> 239     class_count[0] += mask.shape[0]*mask.shape[1] - mask.sum()
    240     class_count[1] += mask.sum()
    241 

ValueError: cannot convert float NaN to integer

Is there any brute-force method to mask the NaNs? What do you suggest to do?

To Reproduce Steps to reproduce the behavior:

1. Go to the U-Net_2D_ZeroCostDL4Mic.ipynb file
2. Copy notebook over own Drive
3. Mount Drive containing two folders: training source and training targets in TIFF format
4. Run all the first cells, unset pretrained option
5. Run cell 4.1, get error

Expected behavior No error, the preparation should run smoothly

Desktop (please complete the following information):

• Archlinux
• Firefox 114.0.1
• Colab Notebook with GPU support

[IvanHCenalmor]

Hi @shoyip ,

Without the data I can not really tell 100%, but I imagine that what is happening is that one of your masks have all the same values in it. This is making the normalizeMinMax function to return NaN values as np.amax(x) - np.amin(x) of an image with the all the values being the same is zero, then the division between zero returns Nan.

# Simple normalization to min/max for the Mask
def normalizeMinMax(x, dtype=np.float32):
  x = x.astype(dtype,copy=False)
  x = (x - np.amin(x)) / (np.amax(x) - np.amin(x))
  return x

Therefore, to solve it, in 021c2c03f5776806fb50b3965b31b0ff13bb1800 I have added a really small value to avoid divisions between zero and with this NaN values.

# Simple normalization to min/max for the Mask
def normalizeMinMax(x, dtype=np.float32):
  x = x.astype(dtype,copy=False)
  x = (x - np.amin(x)) / (np.amax(x) - np.amin(x) + 1e-10)
  return x

Thank you very much for your issue and helping us to solve this problem. Please try the new notebook and if the problem persist, fell free to comment it again in the issue.

Iván