search

neuronflow / BraTS-Toolkit

Code to preprocess, segment, and fuse glioma MRI scans based on the BraTS Toolkit manuscript.
https://www.frontiersin.org/articles/10.3389/fnins.2020.00125/full
GNU Affero General Public License v3.0
72 stars 12 forks source link

BTK Preprocessor stuck on "running HD-Bet" #29

Open c-gamble opened 1 year ago

c-gamble commented 1 year ago

Hello,

There is a closed issue addressing the same problem I am facing, but I figured I would reopen it since the previous issuer seems to no longer be active and the issue was never resolved. When I run BTK single or batch preprocessing, my terminal stays stuck on "running HD-Bet" for more than 8 hours (after which point I simply exited the process). This does not make sense, especially for the single case, considering that HD-Bet's documentation estimates no more than ten seconds per MRI scan. I ran docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi and received the following output.

Screenshot 2023-07-23 at 1 07 22 AM

I'm not actually sure what this command does, but it seems like the output may indicate that no containers are active? Though again, I don't know for sure.

Thank you in advance for any guidance!

neuronflow commented 1 year ago

Hi, Thanks for your interest in BTK and for reporting the problems here. The HD-BET step should take a few minutes max.

I'm not actually sure what this command does, but it seems like the output may indicate that no containers are active? Though again, I don't know for sure.

The comment indicates that docker is installed correctly and can communicate with your GPUs, a common problem for many users but most likely not yours.

Any idea how to replicate your issue? Which mode are you using "gpu" or "cpu"? Did you try both modes?

c-gamble commented 1 year ago

Thank you for your reply! Below is the exact python file I'm running:

from brats_toolkit.preprocessor import Preprocessor

preprocessor = Preprocessor()

exam_name = "BraTS-GLI-[CENSORED]"
t1_file = f"/newresearch/research/projects/Cooper/BraTS_2023/BraTS-Toolkit/preprocessing/input_data/{exam_name}/{exam_name}-t1.nii.gz"
t1c_file = f"/newresearch/research/projects/Cooper/BraTS_2023/BraTS-Toolkit/preprocessing/input_data/{exam_name}/{exam_name}-t1c.nii.gz"
t2_file = f"/newresearch/research/projects/Cooper/BraTS_2023/BraTS-Toolkit/preprocessing/input_data/{exam_name}/{exam_name}-t2.nii.gz"
fla_file = f"/newresearch/research/projects/Cooper/BraTS_2023/BraTS-Toolkit/preprocessing/input_data/{exam_name}/{exam_name}-fla.nii.gz"

output_dir = f"/newresearch/research/projects/Cooper/BraTS_2023/BraTS-Toolkit/segmentation/input_data/{exam_name}"

preprocessor.single_preprocess(
    t1File=t1_file,
    t1cFile=t1c_file,
    t2File=t2_file,
    flaFile=fla_file,
    outputFolder=output_dir,
    mode="gpu",
    confirm=True,
    skipUpdate=False,
    gpuid="2"
)

I've just attempted running the above file, this time removing the "gpuid" line and chaning mode to "cpu," and the program now seems to be stuck at "png slice extraction," as mentioned in #14

neuronflow commented 1 year ago

Okay, maybe your GPUs are too new for HD-BET. Unfortunately, we don't have docker on the cluster where we have the same GPUs, so I have no machine to test and replicate the issue.

Can you double-check the permissions? especially to your output folder?

neuronflow commented 11 months ago

@c-gamble I have an alternative pipeline that you could beta test if you want to :)

neuronflow commented 11 months ago

PS: did you see this post: https://github.com/neuronflow/BraTS-Toolkit/issues/14#issuecomment-886819050 ?