MOTS - Liver dataset preprocess

[Myrrolinz]

Hi! I found MOTS - Liver dataset only provide .nii format data. When I execute python Convert_MOTS_to_nnUNet_dataset.py, it will omit the Liver dataset. How to solve this problem? Could you provide the preprocessed liver dataset or the whole MOTS dataset?

Thank you for your answer!

[Myrrolinz]

and i also encounted this problem:

loading dataset 2024-02-14 20:29:58.642696: Using splits from existing split file: /home/wuyou/data/nnUNet/nnUNet_preprocessed/Task097_11task/splits_final.pkl 2024-02-14 20:29:58.643320: The split file contains 1 splits. 2024-02-14 20:29:58.643412: Desired fold for training: 0 2024-02-14 20:29:58.643524: This split has 2078 training and 527 validation cases. Traceback (most recent call last): File "/home/wuyou/anaconda3/envs/uniseg/bin/nnUNet_train", line 8, in sys.exit(main()) File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/run/run_training.py", line 167, in main trainer.initialize(not validation_only) File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/network_training/UniSeg_Trainer.py", line 128, in initialize self.dl_tr, self.dl_val = self.get_basic_generators() File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/network_training/UniSeg_Trainer.py", line 226, in get_basic_generators self.do_split() File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/network_training/nnUNetTrainerV2.py", line 342, in do_split self.dataset_tr[i] = self.dataset[i] KeyError: 'sub-verse503'

I checked my folder. In nnUNet_raw_data/Task037_VerSe20binary/ and nnUNet_cropped_data/Task037_VerSe20binary/ folder, 'sub-verse503' indeed exist. However in nnUNet_preprocessed/Task037_VerSe20binary/ folder, it doesn't exist. I don't know why this happening :(

Is it possible for you to share the proprocessed data? I know this could be hard but I really need your help! I can provide cloud space >< If you're willing to help me, you can contact me 1195564913@qq.com. thanks!

[yeerwen]

Hello and Happy New Year! The problems you are facing seem to be caused by the use of nnUNet. I think you should check the dataset.json files to ensure they contain the 'sub-verse503' or 'liver'.

[Myrrolinz]

Hello and Happy New Year! The problems you are facing seem to be caused by the use of nnUNet. I think you should check the dataset.json files to ensure they contain the 'sub-verse503' or 'liver'.

May I ask how many CPU core and memory you were using? I think it probably a memory problem. 😭

[yeerwen]

48 CPU core and 125G memory.

[Myrrolinz]

48 CPU core and 125G memory.

I'm experiencing this problem:

2024-02-16 11:54:22.953962: epoch: 0 Traceback (most recent call last): File "/home/wuyou/anaconda3/envs/uniseg/bin/nnUNet_train", line 8, in sys.exit(main()) File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/run/run_training.py", line 183, in main trainer.run_training() File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/network_training/nnUNetTrainerV2.py", line 448, in run_training ret = super().run_training() File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/network_training/nnUNetTrainer.py", line 318, in run_training super(nnUNetTrainer, self).run_training() File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/network_training/network_trainer.py", line 456, in run_training l = self.run_iteration(self.tr_gen, True) File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/network_training/UniSeg_Trainer.py", line 173, in run_iteration data_dict = next(data_generator) File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/dataloading/multi_threaded_augmenter.py", line 218, in next item = self.get_next_item() File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/dataloading/multi_threaded_augmenter.py", line 202, in get_next_item raise RuntimeError("MultiThreadedAugmenter.abort_event was set, something went wrong. Maybe one of " RuntimeError: MultiThreadedAugmenter.abort_event was set, something went wrong. Maybe one of your workers crashed. This is not the actual error message! Look further up your stdout to see what caused the error. Please also check whether your RAM was full Exception in thread Thread-5: Traceback (most recent call last): File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/home/wuyou/anaconda3/envs/uniseg/lib/python3.8/site-packages/nnunet/training/dataloading/multi_threaded_augmenter.py", line 92, in results_loop raise RuntimeError("Abort event was set. So someone died and we should end this madness. \nIMPORTANT: " RuntimeError: Abort event was set. So someone died and we should end this madness. IMPORTANT: This is not the actual error message! Look further up to see what caused the error. Please also check whether your RAM was full

I only have 32G memory. How can I fix this?

[Myrrolinz]

I noticed epoch 0 took me 8 min to finish, then it died. Is it a memory problem? Do I need to expand my memory? btw, I'm using RTX 3060 (12G) now, is that ok?

[yeerwen]

Please tell me how long it takes to train an epoch using the original nnUNet (250 iterations per epoch). If you need about 500 s/epoch, maybe you can update your environment based on https://github.com/MIC-DKFZ/nnUNet/blob/nnunetv1/documentation/expected_epoch_times.md first.