Error in conversion process

[phimoh]

hi :),

first time running pumaz: Unfortunately, already in the conversion process it crashes with the error message below. I tried dicom as well as .nii as input.

Traceback (most recent call last): File "/groups/umcg-mic/tmp02/pmohr/puma_env/bin/pumaz", line 8, in sys.exit(main()) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/pumaz.py", line 132, in main image_conversion.standardize_to_nifti(subject_folder) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/image_conversion.py", line 110, in standardize_to_nifti non_nifti_to_nifti(image_path) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/image_conversion.py", line 58, in non_nifti_to_nifti dicom_info = create_dicom_lookup(input_path) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/image_conversion.py", line 223, in create_dicom_lookup if is_dicom_file(full_path): File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/image_conversion.py", line 190, in is_dicom_file pydicom.dcmread(filename) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pydicom/filereader.py", line 1001, in dcmread fp = open(fp, 'rb') IsADirectoryError: [Errno 21] Is a directory: '/groups/umcg-mic/tmp02/pmohr/puma_env/bin/falcon-linux-x86_64'

[Keyn34]

Hi @phimoh, What does your input directory look like? What OS are you using? I assume Linux?

[phimoh]

Hi @Keyn34 , yes, I run it on a linux cluster.

I have by now done 3 attempts: Attempt 1: Parent_directory/Tracer1/PET_DICOM_DIR Parent_directory/Tracer1/CT_DICOM_DIR and the same for a Tracer2, Tracer3 and Tracer4. But I had other files in the parent_directory, if that matters. Resulted in the error I posted above.

Attempt 2: Three tracers with .nii files starting with CT and PET and directory structure as above. Resulted in Number of puma compliant tracer directories: 0 out of 3 . Not sure why.

Attempt 3: I use three DICOM tracer directories and no other files in the parent directory and it is currently running, at the MOOSE-ing CT files step, looks promising.

[Keyn34]

Hey @phimoh

Attempt 1: Yes, the other files were very likely the problem. We will implement a guard for that.

Attempt 2: The correct PET prefix is PT_. We decided to go for that as this is the corresponding DICOM modality tag. I updated the README.md accordingly!

Attempt 3: Yes, that is how it should be. Please let me know if everything worked out. :)

[phimoh]

Hi short update: I run it on a cluster asking for 1 a40 gpu, but I only asked for 8 hours, and it was not finished yet. I resubmitted asking for 24 hours. Let's see.

What is your experience regarding computation time for e.g. 2 Tracers or 3 Tracers? Does multiplexing increase it significantly or not?

Thanks

[Keyn34]

Very strange @phimoh. Depending on the image size, it should take only 10-15 minutes. Where exactly did it stop?

[phimoh]

Hey @Keyn34, indeed, this time it only run 10 minutes or so, however stopped with an error:

Traceback (most recent call last): File "/groups/umcg-mic/tmp02/pmohr/puma_env/bin/pumaz", line 8, in sys.exit(main()) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/pumaz.py", line 162, in main image_processing.align(puma_dir, ct_dir, pt_dir, mask_dir) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/image_processing.py", line 621, in align copy_reference_image(reference_image, os.path.join(puma_working_dir, constants.ALIGNED_MASK_FOLDER), File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/file_utilities.py", line 355, in copy_reference_image copy_file(source_image, destination_path) File "/groups/umcg-mic/tmp02/pmohr/puma_env/lib/python3.10/site-packages/pumaz/file_utilities.py", line 195, in copy_file shutil.copy(file_path, destination_path) File "/apps/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/shutil.py", line 417, in copy copyfile(src, dst, follow_symlinks=follow_symlinks) File "/apps/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/shutil.py", line 256, in copyfile with open(dst, 'wb') as fdst: FileNotFoundError: [Errno 2] No such file or directory: '/groups/umcg-mic/tmp02/pmohr/dicom_3days_with_multiplex_24h/data/PUMAZ-V01-2024-02-22-09-02-03/aligned_MASK/aligned_MASK_D0_resampled_0_D0_CT_2_quadra_ac_ct_d0_0000.nii.gz'

There is no folder aligned_MASK

Any idea?

[Keyn34]

@phimoh, can you check inside the site-packages folder of the virtual environment where pumaz is installed if there is a bin folder with a greedy folder inside?

[phimoh]

@Keyn34 in bin/ there is no greedy folder, but a falcon folder that has greedy file inside: /groups/umcg-mic/tmp02/pmohr/puma_env/bin/falcon-linux-x86_64/greedy

[Keyn34]

@phimoh Okay, that's odd. This is the first time I have encountered that. But because you are running it on a Linux cluster, that might be something we should look into.

Do you have a different system available to rule out if it's the cluster environment, pumaz, or the data?

[phimoh]

@Keyn34 my colleague has a pc with a gpu. We will try it and update you here the latest on Monday

[LalithShiyam]

Hi @phimoh, great. Please let us know. Actually we can use this as a case to figure out how to run PUMA on a linux cluster. We also had similar issues with falcon before in Leipzig. Your help would highly be appreciated.

[phimoh]

hi @Keyn34 and @LalithShiyam , On the desktop of my colleague: Dicom stopped during the conversion process for some reason.

When we input niftiis it worked. We only needed to skip the setpermission line similarly to falcon due to our environment. We also needed to set KMP_DUPLICATE_LIB_OK=TRUE to resolve one error.

Niftii went through, however, the resulting images are not properly aligned. I will send you an e-mail with more details. I don't know if there are any settings we can test to check if it improves the alignment, for example ignore regions?

[LalithShiyam]

Hi @phimoh, thanks for getting back to us. Sure, please shoot us a mail and regarding the non proper alignment, we need to know where the alignment failed to answer better. cheers, Lalith

[phimoh]

Hi @LalithShiyam, sorry we did a mistake when comparing the images. It seems to work quite nicely, let's discuss further per mail or video call in future

[Keyn34]

@phimoh can we close this, or do you want me to keep this open?

[phimoh]

@phimoh can we close this, or do you want me to keep this open?

Can be closed :)

[Keyn34]

Perfect :)