KITS21 data: Index 3 is out of bounds for axis 1 with size 3

[dannyhow12]

Hi and good day everyone, I encounter the following issue when I tried to train the model from scratch. Pasted below are some error codes displayed.

UnknownError                              Traceback (most recent call last)
[<ipython-input-11-28e4218f17d4>](https://localhost:8080/#) in <module>()
----> 1 model.train(sample_list[30:120], epochs=1)

2 frames
[/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py](https://localhost:8080/#) in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     53     ctx.ensure_initialized()
     54     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 55                                         inputs, attrs, num_outputs)
     56   except core._NotOkStatusException as e:
     57     if name is not None:

UnknownError: Graph execution error:

2 root error(s) found.
  (0) UNKNOWN:  IndexError: index 3 is out of bounds for axis 1 with size 3
Traceback (most recent call last):

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 271, in __call__
    ret = func(*args)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 642, in wrapper
    return func(*args, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 1004, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 830, in wrapped_generator
    for data in generator_fn():

  File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 956, in generator_fn
    yield x[i]

  File "/content/drive/MyDrive/KidneyTumor/MIScnn/miscnn/neural_network/data_generator.py", line 64, in __getitem__
    else : batch = self.generate_batch(idx)

  File "/content/drive/MyDrive/KidneyTumor/MIScnn/miscnn/neural_network/data_generator.py", line 148, in generate_batch
    self.validation)

  File "/content/drive/MyDrive/KidneyTumor/MIScnn/miscnn/processing/preprocessor.py", line 139, in run
    num_classes=1)

  File "/usr/local/lib/python3.7/dist-packages/keras/utils/np_utils.py", line 71, in to_categorical
    categorical[np.arange(n), y] = 1

IndexError: index 3 is out of bounds for axis 1 with size 3

     [[{{node PyFunc}}]]
     [[IteratorGetNext]]
     [[IteratorGetNext/_2]]
  (1) UNKNOWN:  IndexError: index 3 is out of bounds for axis 1 with size 3
Traceback (most recent call last):

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 271, in __call__
    ret = func(*args)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 642, in wrapper
    return func(*args, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 1004, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 830, in wrapped_generator
    for data in generator_fn():

  File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 956, in generator_fn
    yield x[i]

  File "/content/drive/MyDrive/KidneyTumor/MIScnn/miscnn/neural_network/data_generator.py", line 64, in __getitem__
    else : batch = self.generate_batch(idx)

  File "/content/drive/MyDrive/KidneyTumor/MIScnn/miscnn/neural_network/data_generator.py", line 148, in generate_batch
    self.validation)

  File "/content/drive/MyDrive/KidneyTumor/MIScnn/miscnn/processing/preprocessor.py", line 139, in run
    num_classes=1)

  File "/usr/local/lib/python3.7/dist-packages/keras/utils/np_utils.py", line 71, in to_categorical
    categorical[np.arange(n), y] = 1

IndexError: index 3 is out of bounds for axis 1 with size 3

     [[{{node PyFunc}}]]
     [[IteratorGetNext]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_5650]

In order to solve this, I changed num_classes in preprocessor.py to 3 and 1 respectively, where both attempts seem to return the same error as with the original one of num_classes = sample.classes

Here are some notation 1) I am working on the KITS21 dataset, where the imaging.nii.gz and aggregated_MAJ_seg.nii.gz are used. 2) I am using Google Colab to work on this project, but I doubt that the problem is originating from Google Colab as I have successfully cloned this repo to Google Colab.

Thank you

[muellerdo]

Hey @dannyhow12,

mhm, looks like the class annotations in aggregated_MAJ_seg.nii.gz are not ongoing from 0 to num_classes-1.

Tensorflow.keras and also MIScnn requires that the class annotations are from 0 to num_classes-1 (0, 1, 2, 3, ...), because MIScnn utilizes this function for preprocessing: https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical

If you define classes=3, each pixel in the provided segmentation file should be annotated with a class via [0,1,2]. The exception says that he found a pixel with the class annotation 3 which shouldn't exist with 3 classes (due to counting start from zero).

Could you iterate over the annotation files and print out the values via e.g. numpy.unique() to check if they are all from 0 to n-1?

Maybe, the annotation looks like that [1,2,3] instead of [0,1,2]?

Then, you have to either preprocess the annotation files with the standard NIfTI IO interface or just write the class fix code as Subfunction or custom IO interface.

Cheers, Dominik

[dannyhow12]

Hi,

May I ask more specifically on what do you mean by writing the class fix code as Subfunction or custom IO interface?

[muellerdo]

Hello @dannyhow12,

sure! A similar issue is this one: #93

So, the issue is that classes need to be annotated as consecutive values in tensorflow like e.g. [0, 1] for binary segmentation. However, annotations of datasets can sometime be in a non-consecutive fashion.

The standard example is a grayscale mask with [0, 255] for binary segmentation.

So, what we have to do is to replace all 255 annotations to 1, in order to have the consecutive order.

How can we do it this?

You have three options:

• You can do it manually via preprocessing before loading it in MIScnn (iterate over all samples, replace corresponding values to be consecutive, store updated samples to disk again, run MIScnn as usual)
• You can do it during the loading of a sample by writing a custom IO interface for MIScnn (load image, replace corresponding values to be consecutive, return updated NumPy matrix)
• You can do it during preprocessing via a Subfunction (replace corresponding values to be consecutive, return updated NumPy matrix)

Here is an example of a Subfunction to fix a [0,255] annotation to a [0,1] annotation:

The subfunction simply replace the annotation class 255 to 1, which results into a consecutive annotation matrix.

class ChangeValues(Abstract_Subfunction):
   #---------------------------------------------#
   #                Initialization               #
   #---------------------------------------------#
  def __init__(self, val_to_change, change_in):
    self.val_to_change = val_to_change
    self.change_in = change_in
    #---------------------------------------------#
    #                Preprocessing                #
    #---------------------------------------------#
  def preprocessing(self, sample, training=True):
    seg_temp = sample.seg_data
    if(sample.seg_data is not None):
      sample.seg_data = np.where(seg_temp == self.val_to_change, self.change_in, seg_temp)
    #---------------------------------------------#
    #               Postprocessing                #
    #---------------------------------------------#
  def postprocessing(self, sample, prediction, activation_output=False):
    if prediction is not None:
      prediction = np.where(prediction == self.change_in, self.val_to_change, prediction)
    return prediction

You just have to identify with numpy.unique(), now, how the exact class annotation in your dataset look like and then bring it into a consecutive order with e.g. such subfunction.

Hope I was able to help.

Cheers, Dominik

[dannyhow12]

Hello @muellerdo

After reviewing the dataset, I totally missed out on the point that the difference between the KiTS19 and KiTS21 dataset is that there is an extra class of cyst, resulting in some of the CT images with a intensity range from 0 - 3, which are:

• background
• kidney
• tumor
• cyst

Thus, I removed the dataset from it and now it is working. Thank you for pointing out Dominik and stay safe!

Regards, Danny