ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

When I run into X, Y = ImageLoader2D.load_data(img_size, img_size, -1, 'kvasir'), I encountered the following error:

Resizing training images and masks: 1000 0it [00:00, ?it/s] Traceback (most recent call last): File "/home/gufei/rfi_code/DUCK-Net/train.py", line 41, in X, Y = ImageLoader2D.load_data(img_size, img_size, -1, 'kvasir') File "/home/gufei/rfi_code/DUCK-Net/ImageLoader/ImageLoader2D.py", line 64, in loaddata if mask[i, j] >= 127: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Hello,

The error you're encountering "ValueError: The truth value of an array with more than one element is ambiguous" occurs because mask_[i, j] is returning an array instead of a single value. This is happening because the mask image format should be a grayscale image with a depth of 8 bits (the default format of binary segmentation masks). The problem is that most likely the format of the mask image you gave as input has additional dimensions or is saved in the wrong format.

FIX: First, please check the format of the images and masks and make sure they are the exact same format as the ones we provided in the "Data-Sets" section. If they are not, please convert them to the appropriate format before using them.

To help you in finding the problem with the images, I suggest that you print the shape of "mask_" before the "for" loops that contain the condition and make sure that the dimensions are (mask_height, mask_width). If the mask is a grayscale image and it has an extra 3rd dimension, you can remove it by slicing, but make sure not to lose any information while doing so.

Hi, I'm using the Kvasir dataset from your source. After downloading it, I noticed that the mask images have a bit depth of 24 bits. When I print the shape, it shows (352, 352, 3). To make it work, I converted it to grayscale using the following code: mask_ = imread(mask_path, as_gray=True). Is this the correct approach?

Hello,

The error you're encountering "ValueError: The truth value of an array with more than one element is ambiguous" occurs because mask_[i, j] is returning an array instead of a single value. This is happening because the mask image format should be a grayscale image with a depth of 8 bits (the default format of binary segmentation masks). The problem is that most likely the format of the mask image you gave as input has additional dimensions or is saved in the wrong format.

FIX: First, please check the format of the images and masks and make sure they are the exact same format as the ones we provided in the "Data-Sets" section. If they are not, please convert them to the appropriate format before using them.

To help you in finding the problem with the images, I suggest that you print the shape of "mask_" before the "for" loops that contain the condition and make sure that the dimensions are (mask_height, mask_width). If the mask is a grayscale image and it has an extra 3rd dimension, you can remove it by slicing, but make sure not to lose any information while doing so.

Also, I'd like to know what parts of the code I need to modify during training. I noticed that I don't have the 'ProgressFull' and 'ModelSaveTensorFlow' folders. Yesterday, when I trained until the last epoch, I encountered an error as follows:

200/200 [==============================] - 72s 358ms/step - loss: 1.0000 - val_loss: 1.0000 Loss Validation: 1.0 Loss Test: 1.0 Training, epoch 599 Learning Rate: 0.0001 200/200 [==============================] - 72s 358ms/step - loss: 1.0000 - val_loss: 1.0000 Loss Validation: 1.0 Loss Test: 1.0 Loading the model Traceback (most recent call last): File "/home/gufei/rfi_code/DUCK-Net/train.py", line 129, in model = tf.keras.models.load_model(model_path, custom_objects={'dice_metric_loss': dice_metric_loss}) File "/home/gufei/mambaforge/envs/duck_net/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/gufei/mambaforge/envs/duck_net/lib/python3.10/site-packages/keras/saving/save.py", line 226, in load_model raise IOError( OSError: No file or directory found at ModelSaveTensorFlow/kvasir/DuckNet_filters_17_2023-09-14 09:56:24.459396

Could you please advise me on what changes I should make in the code after cloning it to my local machine to successfully train? I'd like to try it with my own dataset. Thank you very much.

Hi, I'm using the Kvasir dataset from your source. After downloading it, I noticed that the mask images have a bit depth of 24 bits. When I print the shape, it shows (352, 352, 3). To make it work, I converted it to grayscale using the following code: mask_ = imread(mask_path, as_gray=True). Is this the correct approach?

Yes, this approach is correct. You're probably reffering to the original datasets, then yes the masks seem to have a bit depth of 24 bits. The datasets split we uploaded already has the files in the right format and ready to be loaded so you can check that out if you wish.

Hello, The error you're encountering "ValueError: The truth value of an array with more than one element is ambiguous" occurs because mask[i, j] is returning an array instead of a single value. This is happening because the mask image format should be a grayscale image with a depth of 8 bits (the default format of binary segmentation masks). The problem is that most likely the format of the mask image you gave as input has additional dimensions or is saved in the wrong format. FIX: First, please check the format of the images and masks and make sure they are the exact same format as the ones we provided in the "Data-Sets" section. If they are not, please convert them to the appropriate format before using them. To help you in finding the problem with the images, I suggest that you print the shape of "mask" before the "for" loops that contain the condition and make sure that the dimensions are (mask_height, mask_width). If the mask is a grayscale image and it has an extra 3rd dimension, you can remove it by slicing, but make sure not to lose any information while doing so.

Also, I'd like to know what parts of the code I need to modify during training. I noticed that I don't have the 'ProgressFull' and 'ModelSaveTensorFlow' folders. Yesterday, when I trained until the last epoch, I encountered an error as follows:

200/200 [==============================] - 72s 358ms/step - loss: 1.0000 - val_loss: 1.0000 Loss Validation: 1.0 Loss Test: 1.0 Training, epoch 599 Learning Rate: 0.0001 200/200 [==============================] - 72s 358ms/step - loss: 1.0000 - val_loss: 1.0000 Loss Validation: 1.0 Loss Test: 1.0 Loading the model Traceback (most recent call last): File "/home/gufei/rfi_code/DUCK-Net/train.py", line 129, in model = tf.keras.models.load_model(model_path, custom_objects={'dice_metric_loss': dice_metric_loss}) File "/home/gufei/mambaforge/envs/duck_net/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/gufei/mambaforge/envs/duck_net/lib/python3.10/site-packages/keras/saving/save.py", line 226, in load_model raise IOError( OSError: No file or directory found at ModelSaveTensorFlow/kvasir/DuckNet_filters_17_2023-09-14 09:56:24.459396

Could you please advise me on what changes I should make in the code after cloning it to my local machine to successfully train? I'd like to try it with my own dataset. Thank you very much.

From the error it seems like the model wasn't properly saved during training and when you try to load it to make the predictions, it doesn't find it. This issue is likely because of the directory structure.

GitHub doesn't allow empty directories to be added to repositories so that's why we couldn't add the whole directory structure for our project. You can either create the "ProgressFull" directory and "ModelSaveTensorFlow" directory with its sub-directories manually, or modify the code to check if they exist and if not to create them automatically. As a guide, your directory structure should look like this: | CustomLayers | ImageLoader | ModelArchitecture | ModelSaveTensorFlow | | kvasir | | cvc-clinicdb | | cvc-colondb | | etis-laribpolypdb | | ADD ANY OTHER DATASET DIRECTORY HERE, MAKE SURE IT HAS THE SAME NAME AS THE ONE IN THE DATA LOADER AND IN THE MODEL NOTEBOOK ("dataset_type" variable) | ProgressFull

Additionally, if you want to train the model on different datasets, I suggest that you first run the training for 1-2 epochs, force the model to save (set "min_loss_for_saving = 1.0") and check that the model was saved properly, and that it generated the progress files.

I hope this helps!

Hi, I'm using the Kvasir dataset from your source. After downloading it, I noticed that the mask images have a bit depth of 24 bits. When I print the shape, it shows (352, 352, 3). To make it work, I converted it to grayscale using the following code: mask_ = imread(mask_path, as_gray=True). Is this the correct approach?

Yes, this approach is correct. You're probably reffering to the original datasets, then yes the masks seem to have a bit depth of 24 bits. The datasets split we uploaded already has the files in the right format and ready to be loaded so you can check that out if you wish.

Hello, The error you're encountering "ValueError: The truth value of an array with more than one element is ambiguous" occurs because mask[i, j] is returning an array instead of a single value. This is happening because the mask image format should be a grayscale image with a depth of 8 bits (the default format of binary segmentation masks). The problem is that most likely the format of the mask image you gave as input has additional dimensions or is saved in the wrong format. FIX: First, please check the format of the images and masks and make sure they are the exact same format as the ones we provided in the "Data-Sets" section. If they are not, please convert them to the appropriate format before using them. To help you in finding the problem with the images, I suggest that you print the shape of "mask" before the "for" loops that contain the condition and make sure that the dimensions are (mask_height, mask_width). If the mask is a grayscale image and it has an extra 3rd dimension, you can remove it by slicing, but make sure not to lose any information while doing so.

Also, I'd like to know what parts of the code I need to modify during training. I noticed that I don't have the 'ProgressFull' and 'ModelSaveTensorFlow' folders. Yesterday, when I trained until the last epoch, I encountered an error as follows: 200/200 [==============================] - 72s 358ms/step - loss: 1.0000 - val_loss: 1.0000 Loss Validation: 1.0 Loss Test: 1.0 Training, epoch 599 Learning Rate: 0.0001 200/200 [==============================] - 72s 358ms/step - loss: 1.0000 - val_loss: 1.0000 Loss Validation: 1.0 Loss Test: 1.0 Loading the model Traceback (most recent call last): File "/home/gufei/rfi_code/DUCK-Net/train.py", line 129, in model = tf.keras.models.load_model(model_path, custom_objects={'dice_metric_loss': dice_metric_loss}) File "/home/gufei/mambaforge/envs/duck_net/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/gufei/mambaforge/envs/duck_net/lib/python3.10/site-packages/keras/saving/save.py", line 226, in load_model raise IOError( OSError: No file or directory found at ModelSaveTensorFlow/kvasir/DuckNet_filters_17_2023-09-14 09:56:24.459396 Could you please advise me on what changes I should make in the code after cloning it to my local machine to successfully train? I'd like to try it with my own dataset. Thank you very much.

From the error it seems like the model wasn't properly saved during training and when you try to load it to make the predictions, it doesn't find it. This issue is likely because of the directory structure.

GitHub doesn't allow empty directories to be added to repositories so that's why we couldn't add the whole directory structure for our project. You can either create the "ProgressFull" directory and "ModelSaveTensorFlow" directory with its sub-directories manually, or modify the code to check if they exist and if not to create them automatically. As a guide, your directory structure should look like this: | CustomLayers | ImageLoader | ModelArchitecture | ModelSaveTensorFlow | | kvasir | | cvc-clinicdb | | cvc-colondb | | etis-laribpolypdb | | ADD ANY OTHER DATASET DIRECTORY HERE, MAKE SURE IT HAS THE SAME NAME AS THE ONE IN THE DATA LOADER AND IN THE MODEL NOTEBOOK ("dataset_type" variable) | ProgressFull

Additionally, if you want to train the model on different datasets, I suggest that you first run the training for 1-2 epochs, force the model to save (set "min_loss_for_saving = 1.0") and check that the model was saved properly, and that it generated the progress files.

I hope this helps!

Thank you very much for your assistance. I can now train and successfully save models.

sir i have converted the masks into grayscale in imageloader as suggested above and i trained the model for 100 epochs but it showed no convergence .Then to diagnose the issue i displayed the loaded image and found that all the maska are totally black and showed no polyp . So this could be the reason of my model not converging. How can i give correct masks as inputs?

RazvanDu / DUCK-Net

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() #4