search

mp-interns / eratosthenes

Intelligent sieving of material candidates for the computationally budget-conscious
BSD 2-Clause "Simplified" License
0 stars 2 forks source link

Discontinuous (integer) search spaces in MOE #15

Open dwinston opened 9 years ago

dwinston commented 9 years ago

I am linking to @HGeerlings issue raised as Yelp/MOE#438. I think flooring is possible, as suggested in Yelp/MOE#369. It appears that SigOpt (the company founded by the creator of MOE) claims to support categorical variables in its service. I don't know how SigOpt does it / whether it's effective.

FYI I set up a small example (via a82497f3200240ea84c1522348f23c38d1d21d5c) to evaluate SigOpt for the water-splitting problem.

dwinston commented 9 years ago

Hmm, SigOpt's "premium" plan (free for 30 days) only allows 150 observations per experiment. The best GA algorithm took ~3000 calculations on average to find the top 20 water-splitting materials (better than ~18,000 for random search!), so unless SigOpt's algorithm is really good, we might have to lean on MOE.

HGeerlings commented 9 years ago

Yeah Neeka and I took a peak at SigOpt's trial deal, and it seems the limitations on it may definitely be too extensive to see if it actually works well. I received an email from Scott, co-founder of SigOpt, that basically summarized a couple of ways (like rounding) too outfit Moe for integers that I can forward to you all.

Does the code you posted 'perov_sigopt' require only a trial of SigOpt?

Thanks, Henry

On Tue, Jul 14, 2015 at 3:09 PM, Donny Winston notifications@github.com wrote:

Hmm, SigOpt's "premium" plan (free for 30 days) only allows 150 observations per experiment. The best GA algorithm took ~3000 calculations on average to find the top 20 water-splitting materials (better than ~18,000 for random search!), so unless SigOpt's algorithm is really good, we might have to lean on MOE.

— Reply to this email directly or view it on GitHub https://github.com/mp-interns/eratosthenes/issues/15#issuecomment-121407423 .

dwinston commented 9 years ago

Yes, the code requires only a trial. I actually invited you and @nmashouf to the trial I started, but you can also start your own.

dwinston commented 9 years ago

Yes, please post the email content here if that makes sense (i.e. no sensitive info), o/w just fwd to us.

HGeerlings commented 9 years ago

From Scott:

Unfortunately, I am no longer actively maintaining the public version of MOE. I'll outline a few options though.

1. You can use SigOpt (sigopt.com http://sigopt.com/), which handles integers (and categories) natively by extending MOE and other methods.

2. You can use the rounding method described above. This may cause repeats as you point out though. You could try to extend MOE to discount these though, or tell MOE that those points are already being sampled, which will also discount their EI. Artificially setting the length scales of those variables to be "integer length" could also help.

3. You can extend MOE to handle these types of parameters natively using a continuous relaxation or other method.

I hope this helps. Let me know if you'd like to try our SigOpt and I can help get you set up.

Thanks Donny,

On Tue, Jul 14, 2015 at 3:21 PM, Donny Winston notifications@github.com wrote:

Yes, please post the email content here if that makes sense (i.e. no sensitive info), o/w just fwd to us.

— Reply to this email directly or view it on GitHub https://github.com/mp-interns/eratosthenes/issues/15#issuecomment-121409966 .