Open rcurtin opened 5 years ago
CMA-ES isn't related to NEAT, but you get to know the optimizer infrastructure.
CNE also does the same as CMAES for point 1. Are there any drawbacks to initializing the population by adding matrices generated by arma::randn() to the starting point?
Right, no need to restrict the starting point.
@zoq For point 2, besides your suggestions, could we also include the initial step size as a parameter? Currently, we experience a problem when we use larger bounds, since the initial step size becomes too large. This causes the CMAES implementation to fail tests when the parameters are changed to include larger bounds (For e.g., [-25,25] for the LogisticRegressionTest). This problem is fixed if we change the step size.
Moreover, I think instead of using one set of bounds for all parameters of a candidate, the user should be able to apply constraints to each parameter separately, since that is more practically useful. Please let me know your thoughts on this
Sounds reasonable to me, I think this is already included in #88?
No, I wanted to make sure it was alright before including it. I'll make the required changes to #88 soon
Sounds good, thanks!
Do you want me to wait to release the next version of ensmallen until #88 is merged?
I'm not sure whether that question was meant for me, but I should let you know that i might not be able to work on #88 much for the next couple weeks, since I'm going to be busy with exams and stuff.
No worries, we can just get #90 merged and then release, assuming that that one is basically ready (I think it is, I'll try and review it tomorrow).
Sure :)
Also, about applying box constraints for each parameter, i don't think we currently have any implementation that does that yet. Would it be fine to add a parameter in the constructor for an arma::cube
where each slice has the constraints? Or is there some other way you would like to do this?
Personally I don't see any issue with an arma::cube
for constraints. It might be worth having that as a separate constructor, so that the user can choose to provide two double
parameters for the minimum and maximum in each dimension (which is easy to make a hypercube with), or a more complex arma::cube
.
This is a continued version of mlpack/mlpack#1475, ported to the ensmallen repository because this is a more appropriate place. This was originally opened by @FloopCZ with the following text:
There are some responses in the original issue about how we should handle each point, but the issue is not yet solved.