Rahul-Narasimhan
commented
4 days ago @saikat-roy to give more context. The inference for the validation cases were all predicted during training and stored as well. But when I try to use the trained model to run inference on an unseen dataset, I get the background workers error. Could you please help ?
Hi Team,
I see that others have faced this issue as well. I was unable to find a solution. The 2d model was trained for 1000 epochs for just fold 0. It ran successfully. I was able to run the best configuration and got the inference instructions.txt as well.
When I tried to use the model to predict on unseen data, i use this command nnUNetv2_predict -d Dataset777_TestData -i /folder1/nnUNet/nnUNet_raw/Dataset777_TestData/imagesTs -o /folder1/nnUNet/nnUNet_raw/Dataset777_TestData/imagesTs_predlowres -f 0 -tr nnUNetTrainer -c 2d -p nnUNetPlans
This is the error I get. Unfortunately, I was not able to figure out what could be the solution from the similar issues that was raised
####################################################################### Please cite the following paper when using nnU-Net: Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2), 203-211. #######################################################################
There are 47 cases in the source folder I am process 0 out of 1 (max process ID is 0, we start counting with 0!) There are 47 cases that I would like to predict Traceback (most recent call last): File "/my-conda-envs/nnunet/bin/nnUNetv2_predict", line 8, in
sys.exit(predict_entry_point())
File "/nnUNet/nnunetv2/inference/predict_from_raw_data.py", line 866, in predict_entry_point
predictor.predict_from_files(args.i, args.o, save_probabilities=args.save_probabilities,
File "/nnUNet/nnunetv2/inference/predict_from_raw_data.py", line 258, in predict_from_files
return self.predict_from_data_iterator(data_iterator, save_probabilities, num_processes_segmentation_export)
File "/nnUNet/nnunetv2/inference/predict_from_raw_data.py", line 351, in predict_from_data_iterator
for preprocessed in data_iterator:
File "/nnUNet/nnunetv2/inference/data_iterators.py", line 111, in preprocessing_iterator_fromfiles
raise RuntimeError('Background workers died. Look for the error message further up! If there is '
RuntimeError: Background workers died. Look for the error message further up! If there is none then your RAM was full and the worker was killed by the OS. Use fewer workers or get more RAM in that case!