MrGiovanni / DiffTumor

[CVPR 2024] Generalizable Tumor Synthesis - Realistic Synthetic Tumors in Liver, Pancreas, and Kidney
https://www.cs.jhu.edu/~alanlab/Pubs24/chen2024towards.pdf
Other
124 stars 9 forks source link

Problem with Singularity #11

Open PedroRASB opened 2 months ago

PedroRASB commented 2 months ago

Hi, I am having some problems inferencing (producing segmentation masks) the Singularity file for DiffTumor, difftumor_final.sif. I am following the procedure in the HuggingFace readme (https://huggingface.co/qicq1c/DiffTumor), and I got a few bugs:

1- When I run the singularity file (singularity run), it tries to install a package, which causes a "Read-only file system" error. The ideal solution for this problem would be to avoid installing any package during singularity run, and intall them only when creating the .sif file (singularity build). My alternative solution was creating an overlay.img file and using it to make the singularity file system not read-only:

sudo singularity overlay create --size 1024 overlay.img
sudo chown user_name:user_name overlay.img
chmod 644 overlay.img
SINGULARITYENV_CUDA_VISIBLE_DEVICES=0 singularity run --fakeroot --overlay overlay.img --nv -B $inputs_data:/workspace/inputs -B $outputs_data:/workspace/outputs difftumor_final.sif

2- After solving problem 1, I got an error when trying to run DiffTumor with multiple CT scans in my input folder ($inputs_data), even after organizing them in the format specified in the readme. The bug is reported in: BugMultipleFiles.txt

My non-ideal solution to this problem was using a temporary directory ($dir) with a single CT scan on it to perform inference:

for ct_file in "$inputs_data"/*.nii.gz; do
  if [ -f "$ct_file" ]; then
    cp "$ct_file" "$dir/$(basename "$ct_file")"
    SINGULARITYENV_CUDA_VISIBLE_DEVICES=0 singularity run --fakeroot --overlay overlay.img --nv -B $dir:/workspace/inputs -B $outputs_data:/workspace/outputs difftumor_final.sif>> "$output_file" 2>&1
    rm "$dir/$(basename "$ct_file")"
  fi
done

3- The code above worked, and I could produce segmentations. However, I still got an error:

Traceback (most recent call last): File "organ_mask_access/organ_test.py", line 135, in main() File "organ_mask_access/organ_test.py", line 132, in main validation(model, test_loader, val_transforms, args) File "organ_mask_access/organ_test.py", line 30, in validation for index, batch in enumerate(tqdm(ValLoader)): File "organ_mask_access/organ_test.py", line 30, in validation for index, batch in enumerate(tqdm(ValLoader)): File "/workspace/anaconda3/lib/python3.8/bdb.py", line 88, in trace_dispatch return self.dispatch_line(frame) File "/workspace/anaconda3/lib/python3.8/bdb.py", line 113, in dispatch_line if self.quitting: raise BdbQuit bdb.BdbQuit

qic999 commented 2 months ago

Hi @PedroRASB. Sorry for these errors. For the "Read-only file system" error, you can convert the singularity container to sandbox. Additionally, I have updated the singularity container on HuggingFace. You can download the latest file and try again.