AIworx-Labs / chocolate

A fully decentralized hyperparameter optimization framework
http://chocolate.readthedocs.io
BSD 3-Clause "New" or "Revised" License
121 stars 41 forks source link

Need possibility for specifying dependent parameters. #8

Open knuds opened 7 years ago

knuds commented 7 years ago

Hi.

We are considering to use chocolate together with genetic algorithms. We have different mutation operators and our hyper parameters is the mutation probability (p1,p2,p3,p4,p5) each between 0.0 and 1.0 but with the constrain that the sum is 1.0. Is it possible to specify such a space ? If yes is would be nice to have such an example in the manual. if no it would be nice to have such a feature implemented.

So long and thanks for chocolate. Knud

fmder commented 7 years ago

It is not possible at the moment to specify this kind of space. Since all implemented algorithms work in a transformed [0, 1]^d space, I guess that you could simply normalize the parameters you receive from the next method. I will skew non linearly the search space, but not irremediably for the search algorithms to be efficient. Don't forget to check for p_i = 0 for all i before normalization.

Please report you success or failure :). We could implement such a method in the available distributions.

knuds commented 7 years ago

I solved my problem by representing the parameter space as the polar angles of a hyper-sphere. Then all the angles is the interval 0 to pi/2. Here is some functions to convert between angles and probabilities.

def ang2prob(ang): prob=np.empty(len(ang)+1)
r=1.0 for i in range(len(ang)): prob[i]=(r*np.cos(ang[i]))*2 r=np.sin(ang[i]) prob[len(ang)]=r**2 return prob

def prob2ang(probs): ang=np.empty(len(probs)-1) for i in range(len(ang)): n=np.sum(probs[i:]) if np.isclose(0.0,n): ang[i]=0.0 else: ang[i]=np.arccos(np.sqrt(probs[i]/n)) return ang

I think enriching chocolate with a way to define a probability vectors like this would be very useful. 1) In genetic algorithms it is a very common optimization challenge. 2) A implementation in chocolate would save the user a lot of converting back and forward.

Knud