Open robkuehl opened 5 years ago
Even before dealing with scale issues, I have some basic questions here.
Are these neural nets connected/correlated in some way? If they are completely separate, I don't see much value in doing that kind of thing. It will negatively affect performance, and won't add any value.
Also, GPyOpt only supports single output objectives. And as you have 5 NNs, it would mean 5 outputs. Are you going to combine them into a single number in some way?
If you are looking for multi-output objectives support, I suggest you take a look at our new library Emukit (https://amzn.github.io/emukit/), which does support this scenario.
I'm currently working with GPyOpt to optimize hyperparameters of a neural network with Bayesian Optimization. I just came to the idea, that you could define the bounds in a way, that you hand over hyperparameters for more than one network to create an ensemble. My question is: For how many hyperparameters / dimension of hyperparameter space does BO does still make sense? For an ensemble with five networks, I already have 85. Nevertheless I know there won't be a finite answer to my question, I would be interested in thoughts regarding it