Skip patient instead of terminate in case of an error

[chris-clem]

Hello,

would it be possible to skip a patient and process the next one in case of an error (e.g. empty CT dir) and not stop the process?

And then maybe in the end you get a list of the patient IDs that failed.

[LalithShiyam]

Hi, Sure, I can write a check for it, but I can’t do so till the weekend. Cheers, Lalith

Lalith Kumar Shiyam Sundar, PhD Post-doktorand Zentrum für Medizinische Physik und Biomedizinische Technik Medizinische Universität Wien, AKH-4L Währinger Gürtel 18-20 A-1090 Wien, Österreich

On 18.11.2022, at 09:03, Christoph Clement @.***> wrote:

Hello,

would it be possible to skip a patient and process the next one in case of an error (e.g. empty CT dir) and not stop the process?

And then maybe in the end you get a list of the patient IDs that failed.

— Reply to this email directly, view it on GitHub https://github.com/QIMP-Team/MOOSE/issues/47, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALSZGN7I7GY4YAEJ3WDFVXDWI4Z4BANCNFSM6AAAAAASEGYU34. You are receiving this because you are subscribed to this thread.

[LalithShiyam]

Hi @chris-clem, just to clarify, what kind of error are we talking about? Because empty CT directory should be ideally detected, and moose exits completely. But I assume you want moose to not exit completely and just proceed with the next patient?

[chris-clem]

Hi,

yes that would be great if MOOSE would continue with the next patient.

Another error that I recently got was the following:

✔ Converted DICOM images in /home/user/Data/.../30122021_5_20211230_170836 to NIFTI
- Only CT data found in folder /home/user/Data/.../30122021_5_20211230_170836, MOOSE will construct non-cerebral tissue atlas (n=37) based on CT data
- Initiating CT segmentation protocols
- CT image to be segmented: /home/user/Data/.../30122021_5_20211230_170836/CT/30122021_5\.CT\.607\.1\.2021\.12\.30\.17\.57\.42\.844000\.4\.0\.1071713859\.IMA_0000\.nii\.gz
✔ Segmented abdominal organs from /home/user/Data/.../30122021_5_20211230_170836/CT/30122021_5.CT.607.1.2021.12.30.17.57.42.844000.4.0.1071713859.IMA_0000.nii.gz
Traceback (most recent call last):
  File "/usr/local/bin/moose", line 133, in <module>
    ct_atlas = ie.segment_ct(ct_file[0], out_dir)
  File "/home/user/Code/MOOSE/src/inferenceEngine.py", line 78, in segment_ct
    out_label = fop.get_files(out_dir, pathlib.Path(nifti_img).stem + '*')[0]
IndexError: list index out of range

It would be great if MOOSE would continue with the next patient and let me know about the errors in the end

[dmnk1308]

Hi @LalithShiyam, I get the same error message for every patient. Is there a fix?

[LalithShiyam]

Hello there, are there any spaces in your folders or something like that? Usually it's the spaces This will be fixed in the next version (coming soon). Let me know if there are no spaces, then we can go to the next possible cause.

[dmnk1308]

Hey, thanks for your fast reply! I can't find any spaces in the file path. The error occurs using your docker image on a linux system.

[LalithShiyam]

Ok, great are you using BOTH pet and ct?

[dmnk1308]

No I'm only using CT images and leave the PET folder empty

[LalithShiyam]

Ok, please don't have an empty PET folder. Remove the empty PET folder all together. You can just have the CT folder alone. Possibly that could be the reason. Please do keep me posted!

[dmnk1308]

Alright I tried it but the error still remains

[LalithShiyam]

Ok, strange. Would you be able to share the anonym data? I can have a look at work and help you out.

[dmnk1308]

Yes, it is a publicly available dataset: https://www.ircad.fr/research/data-sets/respiratory-cycle-3d-ircadb-02/

[LalithShiyam]

Excellent. I will check it out and keep you posted by EOD.

[dmnk1308]

Hey @LalithShiyam, it seems like the problem was on my side. I restarted the system and now it works like a charm. Thanks for your support and for providing the amazing model!

[LalithShiyam]

Fantastic! Happy it worked out well. Let me know if you need further help!

[dmnk1308]

The error occured once again and I was able to find its origin. It seems like the error is thrown if the GPU doesn't have enough memory left.

[LalithShiyam]

Hi there, how big is your GPU? Also do you have enough RAM?

[dmnk1308]

The GPU has 25GB memory and the workstation 64GB RAM

[LalithShiyam]

Gut feeling, the RAM might be the issue, because 64 is not enough for the current version. We found 256 is the RAM size where everything works well. But I understand that it's silly and not practical. We are working on a lighter version, which works with standard workstation configuration. In the mean time, you can swap space (https://www.geeksforgeeks.org/swap-space-in-operating-system/). Slower, but works well. Bear with me, the second version will definitely be lighter and way faster than v1. Don't sweat about the GPU, 25 GB should be fine. I am confident it's the RAM. Because Ubuntu silently kills the process when there is a memory overflow.