search

brainglobe / cellfinder

Automated 3D cell detection in very large images
https://brainglobe.info/documentation/cellfinder/index.html
BSD 3-Clause "New" or "Revised" License
181 stars 39 forks source link

Classification + Analysis step Inefficiencies #356

Closed JuiceSackey closed 1 month ago

JuiceSackey commented 3 years ago

Hello,

It seems like classification and analysis step of the program take entirely too long for my dataset (~1 day for each step). We are analyzing whole brain cfos expression, so there are over 700k cells, which is probably contributing to this issue. I raised an issue on the forums and I was told it may be due to inefficiencies in the coding for these steps and to raise the issue here. The data is all stored/written to an internal HDD.

This is a log file for just analysis ran: cellfinder_2021-09-05_09-30-11_log.txt

This is a log file with the classification+analysis step run: cellfinder_2021-09-07_13-31-22_log.txt

As you can imagine, this is a pretty big bottleneck in processing our data. Please let me know if you need any more information. Thanks!

My specs are as below:

adamltyson commented 3 years ago

I've removed the bug label, as technically everything is working correctly, just very slow on big datasets, but this does need to be improved.

I don't know when I'll get time to look at this, but I would hope it shouldn't be too complex.

adamltyson commented 1 month ago

There have been many changes to the the way cellfinder works since this issue was raised, many of them with the aim of speeding up this type of analysis. In particular, the next version of cellfinder with have GPU support.

I'm going to close this issue, but if problems persist, please let us know.