NOEL-MNI / deepFCD

Automated Detection of Focal Cortical Dysplasia using Deep Learning
https://noel.bic.mni.mcgill.ca/projects/
BSD 3-Clause "New" or "Revised" License
8 stars 3 forks source link

CPU running problems #14

Closed adrianvallsc closed 1 year ago

adrianvallsc commented 1 year ago

I downloaded the project and tried to run it in some nifti files of Cortical Displasias I got. As I do not have a CUDA compatible GPU I tried to run it on my CPU, however when trying to perform the inference step in 1 image, it gets stuck at 0% during at least 5 hours without any advance.

The specifications of my computer are the ones that follow:

MacBook Pro 2020, OSX: Ventura 13, CPU: 2,3 GHz Intel Core i7 4 cores, RAM: 16 GB 3733 MHz LPDDR4X

ravnoor commented 1 year ago

Are you using Docker Desktop for Mac to run the docker image? If so, can you verify the resources allocated to the running container? By default 2G RAM is allocated. See https://docs.docker.com/desktop/settings/mac/ for help

Regarding being stuck at the 0%, does the program exit with an error, or do you manually intervene to exit? Could you try running the program again, and monitor the RAM usage?

adrianvallsc commented 1 year ago

I used Docker to run the docker image, and I increased the allocated RAM memory from 2GB to 8 and to 16 GB, and I also increased the number of cores from 4 to 8. When I monitor the RAM I observe is very RAM intensive, and suddenly the program stops, but the new file is not saved in the folder

ravnoor commented 1 year ago

Is the ${IO_DIRECTORY} folder empty? If not, could you list the contents?

adrianvallsc commented 1 year ago

The structure of the IO directory is:

Finally I could run it in my CPU, but without the docker image (despite I allocated more memory, it dropped out when performing the first step of inference). It took 7661 minutes (5 days and half) to obtain the probability maps. I examined the maps (I guessed the corrects are the ones who are named by _prob_mean_1.nii.gz) but in the case I ran it obtained the maximum probability in the primary motor area.

Captura de pantalla 2023-01-13 a las 8 54 36

ravnoor commented 1 year ago

The output image seems as expected. I would threshold >0.5 to clean up the image a bit, but your assessment about the PMA cluster seems accurate. You can overlay it with the _prob_var_1.nii.gz to get the uncertainty estimate - higher the estimate, larger the uncertainty or lower the confidence).

I'm working on the version wherein you can automatically get a table with all putative clusters (not voxels), their MNI coordinates, probability, rank and uncertainty (or variance). So, look out for a new release in a few days.

Unfortunately, there's no way around the excessive inference time on the CPU.

I can investigate why docker doesn't work. Can you post the output of this command docker images -q noelmni/deep-fcd:latest here? It should be a long alphanumeric string.