search

mp-interns / eratosthenes

Intelligent sieving of material candidates for the computationally budget-conscious
BSD 2-Clause "Simplified" License
0 stars 2 forks source link

Create/Destroy loop for SigOpt #16

Closed nmashouf closed 9 years ago

nmashouf commented 9 years ago

I'm almost done with the wrapper loop for SigOpt, but have run into a problem: each time we create a new experiment, the experiment ID changes so we have to ask for it in some way from the experiment object in order to delete that experiment. The problem is I'm not sure how we can access that using the API (I've been looking through their documentation)! I just emailed them but if anyone has any insight, let me know!

dwinston commented 9 years ago

According to the "create" documentation ( https://sigopt.com/docs/endpoints/experiments/create), you can save the response and peek inside it for the experiment ID.

On Thu, Jul 16, 2015 at 12:28 PM, Neeka Mashouf notifications@github.com wrote:

I'm almost done with the wrapper loop for SigOpt, but have run into a problem: each time we create a new experiment, the experiment ID changes so we have to ask for it in some way from the experiment object in order to delete that experiment. The problem is I'm not sure how we can access that using the API (I've been looking through their documentation)! I just emailed them but if anyone has any insight, let me know!

— Reply to this email directly or view it on GitHub https://github.com/mp-interns/eratosthenes/issues/16.

nmashouf commented 9 years ago

Oh okay, so for the example the experiment id is 1? Thank you! It looks like we can set the id ourselves then?

dwinston commented 9 years ago

Hmm, yeah, looks like you can. :)

dwinston commented 9 years ago

I would still confirm the id from the response, in case there is lag between you destroying an experiment and it actually being destroyed

nmashouf commented 9 years ago

Okay sounds good, thanks! Another note: I'm trying to have A and B be categorical parameters, but it exceeds the "number of categorical values [our] plan allows"! I'll go back to the integer type for A and B.

dwinston commented 9 years ago

Interesting! Yes, pure categorical optimization seems hard. I found a study [1] where they essentially try to convert categorical variables to integer variables with "sorting" schemes.

[1] Pure Categorical Optimization – a Global Descent Approach.pdf

nmashouf commented 9 years ago

Update on loop: Good news is I got it to work! Yay! Bad news is our plan only takes 150 data points, INCLUDING historical data :( so once we transfer the historical data over and try the next data point, we hit our limit.

dwinston commented 9 years ago

Boo. I'll close this issue in that case. Thanks for trying. :)