search

joandre / MCL_spark

An implementation of Markov Clustering algorithm for Spark in Scala
MIT License
34 stars 13 forks source link

Optimize the inflation and expansion parameters #7

Open chungfu27 opened 8 years ago

chungfu27 commented 8 years ago

Hi Joandre,

Do you have any plan to optimize the inflation and expansion parameters automatically? I would like to use genetic algorithm and objective function (modularity) to optimize the inflation and expansion parameters. Do you have any suggestion for this?

joandre commented 8 years ago

Hi chungfu27,

I did not think about it. Actually, I wanted to let the user free of choosing parameters depending on his application.

If you want to do it dynamically (inside MCL), I think it could be dangerous to optimize inflation and expansion parameters during convergence process. Indeed, you could fall on a non steady state and getting blocked by the maximum number of iterations allowed. But clusters you get would not be meaningful.

Anyway, optimizing modularity is a good idea. I would have first used basic grid search parameters tuning (if your problem computation allows such approach).

chungfu27 commented 8 years ago

Hi Joandre, Sorry for so late reply, I think you misunderstand what I mean. I mean I would like to do "hyper parameter optimization for MCL" to select the most suitable inflation/expansion parameters by GA or PSO. Yes, as you said, use basic grid search parameters tuning, but I would like to use meta-heuristic search to optimize these two parameters. I think maybe it will be more efficient than grid search.

joandre commented 8 years ago

Indeed, sorry for my mistake. I understood it later. I did not have time to spend on it during the last weeks and that was not planned at first, but it would be interesting to compare a few methods and add the chosen one as a feature (or even add them all and let users chose). I am open to any pull-request if you want to contribute. Anyway, I will try to take a look if I find the time.