search

RUB-Bioinf / VesselExpress

🔬🖥 Automated Blood Vasculature Analysis of 3D Light-Sheet Image Volumes
GNU General Public License v3.0
37 stars 5 forks source link

Graph analysis took too long to finish #12

Open yliu7366 opened 1 month ago

yliu7366 commented 1 month ago

Dear VesselExpress team,

I ran into issues of being slow when using vesselExpress CLI to analysis heart light sheet images. Seems to me that the graph analysis is running on single thread and it took too long to finish. For example, I gave my task 64 cores but it took more than 120 hours to process the segmented vessels from a sub-volume for the size of 500x8000x8000 (ZYX, voxel size 0.5 um). Is there anyway to make the graph analysis step runs faster?

jxchen01 commented 1 month ago

hi @yliu7366, could you first try to do a test run on a small sample and see if everything works as expected?

@philippa1812 do you have any idea of what could be the reason?

yliu7366 commented 1 month ago

hi @yliu7366, could you first try to do a test run on a small sample and see if everything works as expected?

@philippa1812 do you have any idea of what could be the reason?

The graph analysis step running time seems to scale with the size of vessel network. I was able to finish the whole pipeline on a same sized sub-volume but with less vessel. Even in that case the graph analysis took many hours to finish. I wonder if you have the running time benchmark on your sample datasets for me to compare. Thanks!

philippa1812 commented 1 month ago

Hi @yliu7366

The graph analysis takes about 1 1/2 minutes for an image of 250 MB with 500 x 500 x 501 (XYZ) on standard hardware. The graph analysis has different parts that scale linearly with image size and number of foreground points in the binary and skeleton image. Unfortunately, it does not support parallel processing, so the larger the image and vessel network, the longer the runtime.

yliu7366 commented 1 month ago

Hi @yliu7366

The graph analysis takes about 1 1/2 minutes for an image of 250 MB with 500 x 500 x 501 (XYZ) on standard hardware. The graph analysis has different parts that scale linearly with image size and number of foreground points in the binary and skeleton image. Unfortunately, it does not support parallel processing, so the larger the image and vessel network, the longer the runtime.

Thanks a lot for the data point. Without knowing the details of the graph analysis step, I wonder if loops in the vessel network could make it runs infinitely. I am testing it again with a 25-day limit to see if it will eventually finish on my data.