BUG: vertebrae & bones segmentation model ID

[Nico-Curti]

Hi, I have tried to use the package for bones segmentation and I have initially installed the latest release using pip. Everything works fine, except for the vertebrae segmentation model, which fails with the error:

 File "Python310\Scripts\moosez-script.py", line 33, in <module>
    sys.exit(load_entry_point('moosez', 'console_scripts', 'moosez')())
  File "moosez\moosez.py", line 201, in main
    predict.predict(model_name, input_dir, output_dir, accelerator)
  File "moosez\predict.py", line 78, in predict
    postprocess(original_image_files[0], output_dir, model_name)
  File "moosez\predict.py", line 131, in postprocess
    predicted_image = file_utilities.get_files(output_dir, '.nii.gz')[0]
IndexError: list index out of range

Inspecting the code, the error was due to an un-catched error of the nnUNet_predict command, which fails with the following error:

$ nnUNet_predict -i .\moosez-clin_ct_bones_v1-2023-10-18-13-24-08\CT\temp -o .\moosez-clin_ct_bones_v1-2023-10-18-13-24-08\segmentations -t 111 -m 3d_fullres --fold all --disable_tta

RuntimeError: Could not find a task with the ID 111. Make sure the requested task ID exists and that nnU-Net knows where raw and preprocessed data are located (see Documentation - Installation). Here are your currently defined folders:
nnUNet_preprocessed=None
RESULTS_FOLDER=None
nnUNet_raw_data_base=None
If something is not right, adapt your environemnt variables.

According to the issue 63, I have tried also to downgrade the package version to 2.2.8, trying to use the old version of bone segmentation. However, I get the same issues due to the ID 517 related to this database.

I'm currently working under Windows 10 OS.

Is there something I missed? Thanks in advance

[LalithShiyam]

Hi @Nico-Curti, thanks for the issue! I am not sure if you are interested in v1 bones or v2 vertebrae. Because they are very different! If you want the v1 bones, you can use 2.2.8 but ONLY on a Linux and I see that you are using windows. And nnunetv1 doesn't play well with windows.

Second if you need vertebrae, you can just use it with windows, because the latest moose version 2.2.37 (and every other version except 2.2.8) is based only on nnunetv2.

Kindly let me know.

[Nico-Curti]

I'm mainly interested in the segmentation of the whole skeleton, so I would like to use the v1 bones segmentation but I noticed that good results could be obtained also merging the outputs of vertebrae, ribs, and peripheral_bones. Unfortunately, I have a GPU only under a Windows OS...

I have tried to use the latest release of the package with nnunetv2 and the vertebrae model, but I get the above error due to missing nnunetv2 ID. Is there any way to overcome this issue?

[LalithShiyam]

Ok good, @Nico-Curti kindly tell me the commands you used, also the python version etc., a screenshot of the folder structure would be helpful :)! I will try to reproduce the error and then provide you a solution. Would that be fair?

[Nico-Curti]

I'm working with Python 3.10.4 and I have installed the package using

python -m pip install moosez==2.2.8 --user

As required packages I have

nnunet 1.7.1 nnunetv2 2.2 nibabel 3.2.2 SimpleITK 2.2.1 pydicom 2.2.2 numpy 1.24.4

I'm working with a very simple folder structure as:

nifti
    └── MPC_2_20110413
        ├── CT.nii
        └──  PT.nii

and the error raises using the command

moosez -d ./nifti -m clin_ct_bones_v1

Thanks in advance.

[LalithShiyam]

Hi @Nico-Curti, many thanks - I did look at your inputs, but as I mentioned earlier, moose 2.2.8 will NOT work with windows. And I have no control over it because nnunet v1 is not really compatible with windows. Therefore we inherit all the problems associated it. Unfortunately this version works only on Linux!

Unfortunately moosez 2.2.8 is the only version that can use the v1 models. To make our lives easier, how about we run the stuff using windows WSL? This should solve your problem.

Or you can go through the second route. create a new env, install moosez 2.2.37, and sequentially run the vertebrae, ribs and the peripheral bones models, and combine them together later. Nnunetv2 works well with all the 3 OS, so we have no issues in getting this. Make sure you stick with python 3.10!

Please let me know your thoughts!

[Nico-Curti]

The issue raises also with moosez 2.2.37 but only for cli_ct_vertebrae model. When I run the other models (cli_ct_ribs and cli_ct_peripheral_bones) everything works well. The model cli_ct_vertebrae, instead, fails with the error reported above, related to the ID 111 and due to a missing value in the nnUNet predict command. Probably there are some mistakes in my configuration due to the environment variables or paths, but I didn't find a way to solve it.

Please let me know

[Nico-Curti]

Ok I probably found a solution to make everything work even under the Windows operating system. It's certainly a rough solution but it seems to work on both CPU and GPU, at least for the moosez 2.2.8 version.

I leave the description of the steps here in case it may be useful to someone in the future:

1. Make sure you have successfully installed the moosez 2.2.8 with the nnunet 1.7.1 and nnunetv2 2.2
2. The main differences between the models of v1 and v2 version is related to the usage of nnunet package and the location in which the models will be stored after the download. To properly work with v1 models you need to manually set the environment variable RESULTS_FOLDER equal to the path in which the models\nnUNet_trained_models is created by moosez. The default value (without virtual environment) should be C:\models\nnUNet_trained_models. When you download the v2 model via moosez you should get something like

$ tree C:\\models
└── nnUNet_trained_models
    └── nnUNet
        └── 3d_fullres
            └── Task517_Bones40
                └── nnUNetTrainerV2__nnUNetPlansv2.1
                    ├── all
                    │   ├── debug.json
                    │   ├── model_best.model
                    │   ├── model_best.model.pkl
                    │   ├── model_final_checkpoint.model
                    │   ├── model_final_checkpoint.model.pkl
                    │   ├── postprocessing.json
                    │   ├── progress.png
                    │   ├── training_log_2021_10_20_02_20_54.txt
                    │   └── training_log_2021_10_21_20_38_32.txt
                    ├── fold_0
                    │   ├── debug.json
                    │   ├── model_best.model
                    │   ├── model_best.model.pkl
                    │   ├── progress.png
                    │   └── training_log_2021_10_20_12_37_59.txt
                    └── plans.pkl

3. Now the nnUNet_predict command is able to recognize the ID and paths in which the model checkpoints are stored but there is still an issue in the multithreading management of the model loading. Fortunately this issue was already solved in #1155. You can manually apply the patch installing the pathos package (aka python -m pip install pathos) and editing the file nnunet/inference/predict.py in the site-packages directory according to the instructions used in the PR (aka editing line 24 of the file).

[LalithShiyam]

Ok good, may be we can add this method in FAQ section :)! Thanks a ton! Also, I will try to reproduce your error soonish, we are in a conference and will take sometime to get it done - sorry about that!

[Nico-Curti]

Thank you for the fast answers and I hope this was helpful to others! I'm going to check also if the issue remains with the moosez==2.2.37 combined with the clin_ct_vertebrae under Windows OS asap. The ID problem seems quite close to the clin_ct_bones_v1, but I want to test it before closing this issue, if you agree.

However, the bones v1 models works very well and it is a shame to leave it out of the v2 package.

[LalithShiyam]

Ok let me tell you what, we will get you the same bone regions 🤗 which you had in v1. We will just train it with nnunetv2 and include it in moosev2. Give us a couple of days! We will keep you posted.

@mprires can you take this over? And create a bone model similar to the first moosev1 and include it in moosev2. To make things easier, let's use the OG data for quick turnaround!

[mprires]

Hi! Yes, I will take over this and add a model for bone segmentation similar to the one in moosev1

[mprires]

Hi @Nico-Curti I tried to reproduce the error that you encountered when running the clin_ct_vertebrae model on version 2.2.37 but it worked fine on the Windows machine I have access to. Can you please detail the steps you took to install and run moose when you got that error?

Also happy to tell you that the bone model similar to the one in moosev1 is now being trained, will update you when this is online

[mprires]

Hi @Nico-Curti the bones model of v1 is now online for moosev2. Just use the flag -m clin_ct_all_bones_v1

[LalithShiyam]

Hi @Nico-Curti kindly let us know if this is satisfactory, so that we can close this issue :)!

[Nico-Curti]

Hi @LalithShiyam Sorry for the late replay. Thanks for the new updates and I confirm that the model works well also under Windows OS. I have other issues about the run on multiple patients related to the multi-threading management under Windows, but they are off topics for this issue. I will try to figure out the reasons of these errors and I will let you know. Thanks again for your support!

[LalithShiyam]

Fantastic @Nico-Curti! Will close this!