Open ekalosak opened 4 years ago
Solution appears to be simply deleting that InvalidConfigError
block. Everything is working for me in preliminary testing using mixed bandit and non-bandit inputs.
As far as I remember, this is an intentional choice, because the rest of the codebase has not been developed to handle a mix between bandit and non-bandit space. It may work in some cases but there are just no garantee about the behaviors of surrogate models and acquisition functions.
If you have a strong use case, you are welcome to provide some testcases so that we can know whether it actually performs correctly.
That's correct, indeed we removed that when we built Emukit to avoid those issues (and since the cases of application where minimal).
El vie., 27 mar. 2020 a las 14:51, Zhenwen Dai (notifications@github.com) escribió:
As far as I remember, this is an intentional choice, because the rest of the codebase has not been developed to handle a mix between bandit and non-bandit space. It may work in some cases but there are just no garantee about the behaviors of surrogate models and acquisition functions.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SheffieldML/GPyOpt/issues/310#issuecomment-605042718, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUJF46QJC4UAIZW5OFQ6DDRJS4QRANCNFSM4LUVYFKQ .
It's definitely a desirable functionality - it's actually a blocker for switching to Emukit at the moment. @javiergonzalezh to work this into Emukit, what would you suggest for an implementation strategy? Here are some ideas, obviously non-exhaustive:
@zhenwendai I'm not sure I understand fully - are you aware of any specific spots in the codebase where this mixed bandit/non-bandit input space is a problem? Or is your recommendation out of an abundance of caution? I'll definitely keep this issue updated with concrete results as I get them. It seems like the surrogate models should be perfectly fine - the hyperparameter optimization isn't conditioned on input space restrictions. The acquisition optimizer definitely seems like it'd be an issue, and I'm sure my preliminary concerns listed above are non-exhaustive. Any insight you have on this issue would be very helpful.
Here's a PR supporting the mixed bandit / non-bandit input space: https://github.com/SheffieldML/GPyOpt/pull/313
A mixed input space configured exactly as in (https://github.com/SheffieldML/GPyOpt/blob/master/GPyOpt/core/task/space.py#L22) fails with an
InvalidConfigError
from (https://github.com/SheffieldML/GPyOpt/blob/master/GPyOpt/core/task/space.py#L188).