yuripettorossi
commented
1 day ago Hi! Thank you for the feedbacks!
When it comes to the time needed for the grid search, I think it’s mostly related to the number of simulation performed for each combination of hyperparameters. I’ve set that number at 100, and then I’ve averaged the results of all the simulations, performed with that specific combination, to ensure that the final result was not influenced at all by the random initial state. But in my opinion this was an exaggeration, and setting the number at 10/20 experiments for set is more than enough. This should help to speed up the process. Of course an early stopping, especially for the bigger instances, is also very useful.
I will definitely try to lower the prob parameter as you suggested, thank you for sharing your ideas!
Hello! I like the way you implemented the decreasing in mutation probability to balance exploitation and exploration and the hyperparameter grid search
The first thing I would try to address is that even the first instance takes a long time to run the hyperparameters optimization; instead of having a fixed number of steps at 10_000 for all the instances (except the last 3) you could decrease that value for the first instances, since even from the graph they all seem to converge very fast (At most in ~1000 steps for instance 3). Another direction could be scrapping the fixed num_steps and instead stopping based on some measure of the improvement rate (like an average of fitness increase over the last few steps), especially if you want to optimize for stopping as soon as the improvement slows down without having to manually balance the number of steps.
You could try using a thread pool or some other kind of parallelization for the candidates generation, to increase
num_samplewithout having a big time overhead.The only other thing I would check is the values of
probin the Parameter Grid. From my experiments, I noticed that mutating more than 2-3 genes at a time was disadvantageous - you might want to try even lower rates, like [0.1, 0.3, 0.7], for the last 3 instances as well.