Open lennart81 opened 1 week ago
LalithShiyam
commented
6 days ago Hi @lennart81,please let me know the folder structure. Basically the two tracer directories name and basically the structure. Also please let me know the puma version number.
Just curious, are they also whole body PET datasets?
-Lalith
lennart81
commented
6 days ago I have 2 folders in which I saved a PT***.nii and the according CT***.nii files each. E.g. in folder PSMA-11 I have PT_3_pet_wb_ctac.nii and CT_4_ct_soft_20br383.nii
The scans are from head to thighs - I hope to get some Quadra data when Rod is back in Melbourne ;)
LalithShiyam
commented
6 days ago Again, please let me know the names of the tracer directories. And version of PUMA.
Also do you remove the PUMA generated folder, when you rerun?
lennart81
commented
6 days ago sorry, it was too late. the folders are called "3BP" and "PSMA-11" Version 1.6.6 and yes, I deleted all files and folders PUMA created before re-running
LalithShiyam
commented
6 days ago @lennart81 I looked at your original post and the error during the binary download makes more sense and could likely be the culprit. Do you get this error at the Binary download still?
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['icacls', 'C:\Users\jvhacht\AppData\Local\Programs\Python\bin\beast-binaries-windows-x86_64\greedy.exe', '/grant', 'Everyone:(F)']' returned non-zero exit status 1332.
Hi there,
I have a few cases in which a patient was scanned with 2 tracers in PET/CT. These are not the usual suspects (FDG, PSMA-11 etc.)where a patient got 2 PET/CTs. One is a known tracer the other is experimental. I would like to compare them. Is PUMA able to align the scans so I get an overlay?
Right now, I only get an error message when I try to run PUMA:
Expected anatomical modalities: ['CT'] | Number of required anatomical modalities: 1 | Expected functional modalities:
['PT'] | Number of required functional modalities: 1 | Required prefix for non-DICOM files: ['PT_ for PET', 'CT_ for
CT']
Warning: Any subject datasets in a non-DICOM format that lack the required modalities (as indicated by the file
prefix) will not be included in the analysis.
Multiplexing: False | Custom Colors: False | Segment Tumors: False | Convert to DICOM: False | Regions to Ignore:
['arms', 'legs', 'head']
🌐 BINARIES DOWNLOAD:
Detected system: windows | Detected architecture: x86_64
A local instance of the system specific registration binary has been detected.
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\jvhacht\AppData\Local\Programs\Python\Python311\Scripts\pumaz.exe_main.py", line 7, in
File "C:\Users\jvhacht\AppData\Local\Programs\Python\Python311\Lib\site-packages\pumaz\pumaz.py", line 139, in main
file_utilities.set_permissions(constants.GREEDY_PATH, system_os)
File "C:\Users\jvhacht\AppData\Local\Programs\Python\Python311\Lib\site-packages\pumaz\file_utilities.py", line 64, in set_permissions
raise e
File "C:\Users\jvhacht\AppData\Local\Programs\Python\Python311\Lib\site-packages\pumaz\file_utilities.py", line 56, in set_permissions
subprocess.check_call(["icacls", file_path, "/grant", "Everyone:(F)"], stderr=subprocess.PIPE, stdout=subprocess.PIPE)
File "C:\Users\jvhacht\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['icacls', 'C:\Users\jvhacht\AppData\Local\Programs\Python\bin\beast-binaries-windows-x86_64\greedy.exe', '/grant', 'Everyone:(F)']' returned non-zero exit status 1332.
🚀 RUNNING PREPROCESSING AND REGISTRATION PIPELINE:
Preprocessing PUMA compliant subjects ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
CUDA is available with 1 GPU(s). Predictions will be run on GPU.
MOOSE-ing CT files | Model: clin_ct_body | Accelerator: cuda ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% [2/2] • Time elapsed: 0:00:46 • CPU Load: 11.2% • Memory Load: 3.2% • GPU Load: 0.0%
MOOSE-ing CT files | Model: clin_ct_PUMA | Accelerator: cuda ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% [2/2] • Time elapsed: 0:03:31 • CPU Load: 17.2% • Memory Load: 3.2% • GPU Load: 0.0%
Aligning images... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% [1/1] • Time elapsed: 0:03:05 • CPU Load: 59.1% • Memory Load: 2.0%
Traceback (most recent call last):
File "/data/jvhacht/puma-env/bin/pumaz", line 8, in
sys.exit(main())
File "/data/jvhacht/puma-env/lib/python3.10/site-packages/pumaz/pumaz.py", line 185, in main
reference_img = image_processing.align(puma_dir, ct_dir, pt_dir, mask_dir)
File "/data/jvhacht/puma-env/lib/python3.10/site-packages/pumaz/image_processing.py", line 664, in align
modality_image = find_corresponding_image(modality_dir, os.path.basename(moving_image))
File "/data/jvhacht/puma-env/lib/python3.10/site-packages/pumaz/image_processing.py", line 616, in find_corresponding_image
raise FileNotFoundError(f"No corresponding image found in {modality_dir} for {reference_basename}")
FileNotFoundError: No corresponding image found in /mnt/I/TotalSegmentator/PUMA/PUMAZ-v1-2024-09-25-20-14-33/PT for resampled_1_PSMA-11_CT_PSMA_0000.nii.gz@mprires @Keyn34 we did maul with this windows error right? I think one of you had a solution with Windows issues. Would you be kind enough to look into this?
lennart81
commented
6 days ago The original error occured on my win notebook. However, I switched to our linux server and there I only get the new error after alignment was completed.
Keyn34
commented
6 days ago @lennart81, regarding the Windows problem: could you please navigate to puma-env/lib/python3.10/site-packages/pumaz and then search for the file_utilities.py file and open it?
Once you are there, exchange lines 55 and 56 with
if system_type == "windows":
subprocess.check_call(["icacls", file_path, "/grant", "*S-1-1-0:(F)"])It would also be helpful, as @LalithShiyam pointed out, to get a complete view of your directory tree of the subject folder to confirm that the structure is correct. You are probably not allowed to share your data, right? Because this would be the easiest way to find out whats happening.
lennart81
commented
6 days ago @lennart81, regarding the Windows problem: could you please navigate to
puma-env/lib/python3.10/site-packages/pumazand then search for thefile_utilities.pyfile and open it?Once you are there, exchange lines 55 and 56 with
if system_type == "windows": subprocess.check_call(["icacls", file_path, "/grant", "*S-1-1-0:(F)"])It would also be helpful, as @LalithShiyam pointed out, to get a complete view of your directory tree of the subject folder to confirm that the structure is correct. You are probably not allowed to share your data, right? Because this would be the easiest way to find out whats happening.
Did that - PUMA started. This will take a while on my office notebook ;)
This is the file/folder structure: Directory of I:\Segmentator\PUMA
26.09.2024 12:01
Directory of I:\Segmentator\PUMA\3BP
26.09.2024 09:53
Directory of I:\Segmentator\PUMA\PSMA-11
26.09.2024 09:53
LalithShiyam
commented
6 days ago Can you share the nifti, just for us to figure out quickly what's going wrong?
LalithShiyam
commented
6 days ago OK, I just renamed a dummy dataset, to your file names and PUMA seems to work. @lennart81 do me a favor, and just run it again, let it crash and please show me the screenshot of all the intermediate files generated by PUMA.
lennart81
commented
6 days ago I just realized I only tried this with one pair of PT/CT files converted to NII files earlier in moose. I'll export the original DICOM files and give it a try. Give me a few minutes, I'm currently in a GCP training
lennart81
commented
6 days ago it worked under windows (~40 mins)! thanks @Keyn34 I'll try to run it again under Linux with the DICOMs
lennart81
commented
6 days ago OK, I just renamed a dummy dataset, to your file names and PUMA seems to work. @lennart81 do me a favor, and just run it again, let it crash and please show me the screenshot of all the intermediate files generated by PUMA.
I copied the original DICOMs to my folder and ran it again. It crashed again and these are all the files in the PUMAZ folder:
Volume in drive I is Data Volume Serial Number is C472-A7D5
Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16
26.09.2024 13:18
Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\body_masks
26.09.2024 13:15
Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\CT
26.09.2024 13:15
Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\PT
26.09.2024 13:10
Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\puma_masks
26.09.2024 13:18
Files Listed:
19 File(s) 983,345,763 bytes
14 Dir(s) 9,484,152,553,472 bytes free The same datasets run in windows? DICOM? same folder structure and everything?
Because the PT folder doesn't have the right naming convention. If it doesn't work here, it shouldn't work in windows too.
@Keyn34 @mprires
mprires
commented
6 days ago Hi @lennart81, It appears the issue may stem from the renaming of files and masks. Could you please clarify a couple of details? Was this run executed on a Windows system? If so, could you specify what changes were made compared to the previous run that was successful?
OK, I just renamed a dummy dataset, to your file names and PUMA seems to work. @lennart81 do me a favor, and just run it again, let it crash and please show me the screenshot of all the intermediate files generated by PUMA.
I copied the original DICOMs to my folder and ran it again. It crashed again and these are all the files in the PUMAZ folder:
Volume in drive I is Data Volume Serial Number is C472-A7D5
Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16
26.09.2024 13:18
. 26.09.2024 13:10 .. 26.09.2024 13:18 1,899,832 aligned_MASK_resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz 26.09.2024 13:15 body_masks 26.09.2024 13:15 CT 26.09.2024 13:10 PT 26.09.2024 13:18 puma_masks 1 File(s) 1,932,600 bytes Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\body_masks
26.09.2024 13:15
. 26.09.2024 13:18 .. 26.09.2024 13:11 364 dataset.json 26.09.2024 13:11 7,165 plans.json 26.09.2024 13:11 538 predict_from_raw_data_args.json 26.09.2024 13:15 890,827 resampled_0_S1_CT_4_ct_soft_20br383_0000.nii.gz 26.09.2024 13:15 886,999 resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz 5 File(s) 1,785,893 bytes Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\CT
26.09.2024 13:15
. 26.09.2024 13:18 .. 26.09.2024 13:10 63,238,867 resampled_0_S1_CT_4_ct_soft_20br383_0000.nii.gz 26.09.2024 13:11 62,041,090 resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz 2 File(s) 125,279,957 bytes Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\PT
26.09.2024 13:10
. 26.09.2024 13:18 .. 26.09.2024 13:10 406,560,352 PT_0_3_pet_wb_ctac.nii 26.09.2024 13:10 402,688,352 PT_1_3_pet_wb_ctac.nii 2 File(s) 809,248,704 bytes Directory of I:\Segmentator\PUMA\PUMAZ-v1-2024-09-26-13-06-16\puma_masks
26.09.2024 13:18
. 26.09.2024 13:18 .. 26.09.2024 13:13 931 dataset.json 26.09.2024 13:13 15,978 plans.json 26.09.2024 13:13 548 predict_from_raw_data_args.json 26.09.2024 13:15 912,359 resampled_0_S1_CT_4_ct_soft_20br383_0000.nii.gz 26.09.2024 13:15 905,697 resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz 26.09.2024 13:15 126 resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz_affine.mat 26.09.2024 13:18 21,689,938 resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz_inverse_warp.nii.gz 26.09.2024 13:15 57 resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz_moment.mat 26.09.2024 13:17 21,572,975 resampled_1_S2_CT_4_ct_soft_20br383_0000.nii.gz_warp.nii.gz 9 File(s) 45,098,609 bytes
Files Listed: 19 File(s) 983,345,763 bytes 14 Dir(s) 9,484,152,553,472 bytes free
LalithShiyam
commented
6 days ago @mprires : its linux. look at the slash.
lennart81
commented
6 days ago I had one successful Windows run but could not use the same files on Linux - I always get the same "no corresponding image found..." error
lennart81
commented
6 days ago do you have any test files (PET/CT) I can download an run?
LalithShiyam
commented
6 days ago I had one successful Windows run but could not use the same files on Linux - I always get the same "no corresponding image found..." error
Were they the same patient files that you tried on windows? Meaning same patient was tested on windows and Linux.
and unfortunately, we can't share any of the images, because they are DTA protected. If the images that you ran are from MEL, you can transfer them, because we do have a DTA and I can talk to Rod.
LalithShiyam
commented
6 days ago I had one successful Windows run but could not use the same files on Linux - I always get the same "no corresponding image found..." error
The error seems clear: the PT file is being named incorrectly. It should start with the tracer_dir name and it is not being named so. It should be S1_PT_0_3_pet_wb_ctac.nii instead of PT_0_3_pet_wb_ctac.nii. Since the renaming relies on the DICOM header, it's difficult to determine the cause. Unfortunately, we haven't been able to reproduce the error on our side. If sharing the data is not possible, there's not much we can do remotely; however, the easiest solution might be to use Anydesk for further troubleshooting.
Sorry about this.
lennart81
commented
5 days ago I had a look at the files PUMA creates. There is a discrepancy between Win and Linux outputs On Windows PUMA created a folder called CT in which both CTs can befound named resampled_X_folder-name_CT_0000.nii.gz On Linux: folder-name_resampled__X_folder-name_CT_0000.ni.gz
Should it be that way?
Keyn34
commented
5 days ago No, it should not. And I have never seen this before.
What is the input of PUMA? DICOM? Try to use nifti right away. I think it's from renaming the files with the re module of Python after the DICOM conversion.
Hi there,
I have a few cases where a patient got 2 PET/CTs. One is a known tracer the other is experimental. I would like to compare them. Is PUMA able to align the scans so I get an overlay?
Right now, I only get an error message when I try to run PUMA:
🚀 RUNNING PREPROCESSING AND REGISTRATION PIPELINE:
Preprocessing PUMA compliant subjects ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 CUDA is available with 1 GPU(s). Predictions will be run on GPU. MOOSE-ing CT files | Model: clin_ct_body | Accelerator: cuda ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% [2/2] • Time elapsed: 0:00:46 • CPU Load: 11.2% • Memory Load: 3.2% • GPU Load: 0.0% MOOSE-ing CT files | Model: clin_ct_PUMA | Accelerator: cuda ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% [2/2] • Time elapsed: 0:03:31 • CPU Load: 17.2% • Memory Load: 3.2% • GPU Load: 0.0% Aligning images... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% [1/1] • Time elapsed: 0:03:05 • CPU Load: 59.1% • Memory Load: 2.0% Traceback (most recent call last): File "/data/jvhacht/puma-env/bin/pumaz", line 8, in
sys.exit(main())
File "/data/jvhacht/puma-env/lib/python3.10/site-packages/pumaz/pumaz.py", line 185, in main
reference_img = image_processing.align(puma_dir, ct_dir, pt_dir, mask_dir)
File "/data/jvhacht/puma-env/lib/python3.10/site-packages/pumaz/image_processing.py", line 664, in align
modality_image = find_corresponding_image(modality_dir, os.path.basename(moving_image))
File "/data/jvhacht/puma-env/lib/python3.10/site-packages/pumaz/image_processing.py", line 616, in find_corresponding_image
raise FileNotFoundError(f"No corresponding image found in {modality_dir} for {reference_basename}")
FileNotFoundError: No corresponding image found in /mnt/I/TotalSegmentator/PUMA/PUMAZ-v1-2024-09-25-20-14-33/PT for resampled_1_PSMA-11_CT_PSMA_0000.nii.gz