2d numpy images

[eyalhana]

Hi, First of all, thank you for your fantastic work! Each image in my dataset is of size [H,W] in numpy format.

How should I save the images in the raw folder?

If I save it as nii.gz, should I add a dimension [1,H,W] or two dimensions [1,1,H,W] using np.newaxis, before using nib.Nifti1Image(img, np.eye(4))?

I would appreciate an explanation of how I should convert the images and how to add dimensions before saving the images. The inputs size is [H,W] and the original format is numpy.

Thank you!

[FabianIsensee]

Hey, you don't need to mess with dimension. Unless i messed up (let me know in that case) you can just save your image as 2D niftis. If you prefer png (and your pixel type is compatible with that) then that's fine, too. Best, Fabian

[eyalhana]

Thanks for the answer! After saving the [H,W] numpy files using the following command: nib_img = nib.Nifti1Image(np_img, np.eye(4)) nib.save(nib_img,filename_img)

I have got the following error while trying to run nnUNetv2_plan_and_preprocess -d 500 -c 2d --verify_dataset_integrity

Failed to open file /home/eyalhan/nnUNetT1/nnUNet_raw/Dataset500_cardiac/imagesTr/cardiac_00000_0000.nii.gz with reader <class 'nnunetv2.imageio.simpleitk_reader_writer.SimpleITKIO'>:
...
AssertionError: only 3d images are supported by NibabelIO
...
RuntimeError: Unable to determine a reader for file ending .nii.gz and file /home/eyalhan/nnUNetT1/nnUNet_raw/Dataset500_cardiac/imagesTr/cardiac_00000_0000.nii.gz (file None means no file provided).

After adding "overwrite_image_reader_writer": "SimpleITKIO" to the json file I got the following:

Using <class 'nnunetv2.imageio.simpleitk_reader_writer.SimpleITKIO'> reader/writer
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/verify_dataset_integrity.py", line 33, in verify_labels
    seg, properties = rw.read_seg(label_file)
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/imageio/simpleitk_reader_writer.py", line 115, in read_seg
    return self.read_images((seg_fname, ))
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/imageio/simpleitk_reader_writer.py", line 59, in read_images
    spacings_for_nnunet[-1] = list(np.abs(spacings_for_nnunet[-1]))
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/eyalhan/.local/bin/nnUNetv2_plan_and_preprocess", line 8, in <module>
    sys.exit(plan_and_preprocess_entry())
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/plan_and_preprocess_entrypoints.py", line 173, in plan_and_preprocess_entry
    extract_fingerprints(args.d, args.fpe, args.npfp, args.verify_dataset_integrity, args.clean, args.verbose)
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/plan_and_preprocess_api.py", line 46, in extract_fingerprints
    extract_fingerprint_dataset(d, fingerprint_extractor_class, num_processes, check_dataset_integrity, clean,
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/plan_and_preprocess_api.py", line 29, in extract_fingerprint_dataset
    verify_dataset_integrity(join(nnUNet_raw, dataset_name), num_processes)
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/verify_dataset_integrity.py", line 186, in verify_dataset_integrity
    result = p.starmap(
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 372, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

I would appreciate an explanation what I should do :-) Thanks!

[FabianIsensee]

OK so I guess I messed up after all haha. Sorry for that If you can share the file then I can work on a solution :-) Best, Fabian

[eyalhana]

I really appreciate it! thanks!!! :-)

I'm not sure which files you mean, so for now I'll share here a short script that creates random data in the same format without forcing you to mess with the original dataset..

import numpy as np
import nibabel as nib
import os
import matplotlib.pyplot as plt

# ***************** Loading *****************

out_base = '/home/eyalhan/nnUNetT1'
path = '/home/eyalhan/nnUNetT1/nnUNet_raw/Dataset501_cardiac'
os.makedirs(path,exist_ok=True)
os.makedirs(path+'/imagesTr',exist_ok=True)
os.makedirs(path+'/labelsTr',exist_ok=True)

ind = 0 
for p in range(10):
    np_img = np.random.rand(160,160).astype(np.float64)
    np_seg = np.random.randint(0,2,(160,160)).astype(np_img.dtype)

    nib_img = nib.Nifti1Image(np_img, np.eye(4))
    nib_seg = nib.Nifti1Image(np_seg, np.eye(4))

    filename_img  = os.path.join(path,'imagesTr', 'cardiac_'+str(ind).zfill(5)+'_0000.nii.gz')
    nib.save(nib_img,filename_img)  

    filename_seg  = os.path.join(path,'labelsTr', 'cardiac_'+str(ind).zfill(5)+'.nii.gz')
    nib.save(nib_seg,filename_seg)  

    ind = ind+1

dataset.json :

{ 
 "channel_names": {  
   "0": "T1"
 }, 
 "labels": {  
   "background": 0,
   "myocardial": 1
 }, 
 "numTraining": 10, 
 "file_ending": ".nii.gz",
 "overwrite_image_reader_writer": "SimpleITKIO" 
 }

[FabianIsensee]

Hey, I mean this file: /home/eyalhan/nnUNetT1/nnUNet_raw/Dataset500_cardiac/imagesTr/cardiac_00000_0000.nii.gz Best, Fabian

[eyalhana]

Thanks!

Here are some of the files: cardiac_00002.nii.gz cardiac_00001.nii.gz cardiac_00000.nii.gz cardiac_00002_0000.nii.gz cardiac_00001_0000.nii.gz cardiac_00000_0000.nii.gz

BTW the data was taken from https://cardiacmr.hms.harvard.edu/downloads-0 (After performing several operations such as creating the segmentation and cutting the relevant area and converting the files to np from mat, etc.)

[FabianIsensee]

Thanks for sharing! It was a bug and is fixed now. Please install latest version from Github :-)

[eyalhana]

Thanks you!

Unfortunately I think that maybe I miss something. After perform pip uninstall nnunetv2 pip install nnunetv2 nnUNetv2_plan_and_preprocess -d 500 -c 2d --verify_dataset_integrity

I get the following message:

(nnUNet) [eyalhan@bm.technion.ac.il@argus02 ~]$ nnUNetv2_plan_and_preprocess -d 500 -c 2d --verify_dataset_integrity
Fingerprint extraction...
Dataset500_cardiac
Using <class 'nnunetv2.imageio.simpleitk_reader_writer.SimpleITKIO'> reader/writer
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/verify_dataset_integrity.py", line 33, in verify_labels
    seg, properties = rw.read_seg(label_file)
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/imageio/simpleitk_reader_writer.py", line 115, in read_seg
    return self.read_images((seg_fname, ))
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/imageio/simpleitk_reader_writer.py", line 59, in read_images
    spacings_for_nnunet[-1] = list(np.abs(spacings_for_nnunet[-1]))
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/eyalhan/.local/bin/nnUNetv2_plan_and_preprocess", line 8, in <module>
    sys.exit(plan_and_preprocess_entry())
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/plan_and_preprocess_entrypoints.py", line 173, in plan_and_preprocess_entry
    extract_fingerprints(args.d, args.fpe, args.npfp, args.verify_dataset_integrity, args.clean, args.verbose)
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/plan_and_preprocess_api.py", line 46, in extract_fingerprints
    extract_fingerprint_dataset(d, fingerprint_extractor_class, num_processes, check_dataset_integrity, clean,
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/plan_and_preprocess_api.py", line 29, in extract_fingerprint_dataset
    verify_dataset_integrity(join(nnUNet_raw, dataset_name), num_processes)
  File "/home/eyalhan/.local/lib/python3.9/site-packages/nnunetv2/experiment_planning/verify_dataset_integrity.py", line 186, in verify_dataset_integrity
    result = p.starmap(
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 372, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/tcmldrive/lib/envs/nnUNet/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

Do you have idea what I should do?

Thanks!

[FabianIsensee]

Please install latest version from Github :-)

git clone https://github.com/MIC-DKFZ/nnUNet.git
cd nnUNet
pip install -e .

[eyalhana]

Thanks, Sorry for all the questions.

Now I have a new error..

####################
verify_dataset_integrity Done. 
If you didn't see any error messages then your dataset is most likely OK!
####################

Experiment planning...
2D U-Net configuration:
{'data_identifier': 'nnUNetPlans_2d', 'preprocessor_name': 'DefaultPreprocessor', 'batch_size': 8, 'patch_size': array([160, 160]), 'median_image_size_in_voxels': array([160., 160.]), 'spacing': array([1., 1.]), 'normalization_schemes': ['ZScoreNormalization'], 'use_mask_for_norm': [False], 'UNet_class_name': 'PlainConvUNet', 'UNet_base_num_features': 32, 'n_conv_per_stage_encoder': (2, 2, 2, 2, 2, 2), 'n_conv_per_stage_decoder': (2, 2, 2, 2, 2), 'num_pool_per_axis': [5, 5], 'pool_op_kernel_sizes': [[1, 1], [2, 2], [2, 2], [2, 2], [2, 2], [2, 2]], 'conv_kernel_sizes': [[3, 3], [3, 3], [3, 3], [3, 3], [3, 3], [3, 3]], 'unet_max_num_features': 512, 'resampling_fn_data': 'resample_data_or_seg_to_shape', 'resampling_fn_seg': 'resample_data_or_seg_to_shape', 'resampling_fn_data_kwargs': {'is_seg': False, 'order': 3, 'order_z': 0, 'force_separate_z': None}, 'resampling_fn_seg_kwargs': {'is_seg': True, 'order': 1, 'order_z': 0, 'force_separate_z': None}, 'resampling_fn_probabilities': 'resample_data_or_seg_to_shape', 'resampling_fn_probabilities_kwargs': {'is_seg': False, 'order': 1, 'order_z': 0, 'force_separate_z': None}, 'batch_dice': True}

Using <class 'nnunetv2.imageio.simpleitk_reader_writer.SimpleITKIO'> reader/writer
Plans were saved to /home/eyalhan/nnUNetT1/nnUNet_preprocessed/Dataset500_cardiac/nnUNetPlans.json
Preprocessing...
Traceback (most recent call last):
  File "/home/eyalhan/.local/bin/nnUNetv2_plan_and_preprocess", line 33, in <module>
    sys.exit(load_entry_point('nnunetv2', 'console_scripts', 'nnUNetv2_plan_and_preprocess')())
  File "/home/eyalhan/nnUNetT1/nnUNet/nnunetv2/experiment_planning/plan_and_preprocess_entrypoints.py", line 182, in plan_and_preprocess_entry
    preprocess(args.d, args.overwrite_plans_name, args.c, args.np, args.verbose)
  File "/home/eyalhan/nnUNetT1/nnUNet/nnunetv2/experiment_planning/plan_and_preprocess_api.py", line 127, in preprocess
    preprocess_dataset(d, plans_identifier, configurations, num_processes, verbose)
  File "/home/eyalhan/nnUNetT1/nnUNet/nnunetv2/experiment_planning/plan_and_preprocess_api.py", line 96, in preprocess_dataset
    raise RuntimeError(
RuntimeError: The list provided with num_processes must either have len 1 or as many elements as there are configurations (see --help). Number of configurations: 1, length of num_processes: 3

I didn't change the num_processes (and I am not sure if and how to do it..)

[FabianIsensee]

My bad. I will take a closer look later. Until then use -np 8 when running planning and preprocessing

[eyalhana]

Thank you, it works!

I really appreciate your availability and the effort to answer questions quickly even though I assume you are very busy.

Have a great day, Eyal

[FabianIsensee]

You, too :-)