search

BrutishGuy / pyfriends

Python implementation of the paper "PyFriends: The First Fully Generalized Friends-of-Friends Extragalactic Galaxy Group Finder", using a Friends-of-Friends (FoF) algorithm for galaxy group detection, augmented by graph theory approaches.
3 stars 1 forks source link

Overall Speed of the initial Algorithm takes way too long. #3

Open TrystanScottLambert opened 4 years ago

TrystanScottLambert commented 4 years ago

So far for 45 000 galaxies the algorithm takes about 10-15 minutes to run 10 times. Ideally at least 100 runs would be done meaning about 100 minutes.

Need to speed up the overall time of the actually FoF algorithm.

Obvious solution is to parallelize running the different trials but I think we can speed up the whole thing before we parallelize and then future proof using parallelization.

For 45 000 galaxies, the time taken should be under 2 mins for 100 runs. This would mean future large surveys with 100's of thousands of galaxies would be handled well.

I don't want to use C :'(

TrystanScottLambert commented 4 years ago

Just tried the Algorithm on a data set 2x larger than the 2MRS. One run takes about 1.5 minutes on my machine whereas the 2MRS takes 1 minute. Whole 90 000 galaxy data set was processed in 17 minutes. If we can take this down even more we should.