search

Yelp / MOE

A global, black box optimization engine for real world metric optimization.
Other
1.31k stars 140 forks source link

moe.tests.views.rest.gp_next_points_test TestGpNextPointsViews.test_interface_returns_same_as_cpp extremely slow #374

Closed suntzu86 closed 10 years ago

suntzu86 commented 10 years ago

Before this push: https://github.com/Yelp/MOE/pull/352, i.e., in 9345f69dcd094e2febf3c4688d27cbc8575f713d:

moe.tests.views.rest.gp_next_points_test TestGpNextPointsViews.test_interface_returns_same_as_cpp ... ok in 0.07s

After, i.e., in c63f5c03e5763dae1f1a9d9551c2fb5c0ac24e29 performance varies widely but it's some 300x slower:

moe.tests.views.rest.gp_next_points_test TestGpNextPointsViews.test_interface_returns_same_as_cpp ... ok in 47.54s

moe.tests.views.rest.gp_next_points_test TestGpNextPointsViews.test_interface_returns_same_as_cpp ... ok in 23.45s

I'd guess that means somewhere in here we're doing full solves instead of very limited solves with the "TEST_" parameters. But I'm honestly not sure. Since the parameters are set in the test file, that might mean we're no longer parsing optimizer parameters correctly? Don't know.

suntzu86 commented 10 years ago

Still quite a bit slower than originally, but that doesn't seem avoidable due to the expense of even low accuracy mvndst calls.

After @bulutcocuk pushed #381, we're down to:

moe.tests.views.rest.gp_next_points_test TestGpNextPointsViews.test_interface_returns_same_as_cpp ... ok in 1.56s

which is ok