Optimizer interface changed

[AlexanderFabisch]

Currently CI runs fail with the following errors.

======================================================================
ERROR: test_black_box_search.test_black_box_search_from_dicts
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/behavior_search/test/test_black_box_search.py", line 24, in test_black_box_search_from_dicts
    bs.init(5, 5)
  File "/root/project/bolero-dev/learning/bolero/bolero/behavior_search/black_box_search.py", line 97, in init
    self.optimizer = _initialize_optimizer(self.optimizer, self.behavior)
  File "/root/project/bolero-dev/learning/bolero/bolero/behavior_search/black_box_search.py", line 38, in _initialize_optimizer
    optimizer = from_dict(optimizer)
  File "/root/project/bolero-dev/learning/bolero/bolero/utils/module_loader.py", line 181, in from_dict
    return _from_dict(name, config)
  File "/root/project/bolero-dev/learning/bolero/bolero/utils/module_loader.py", line 247, in _from_dict
    "'%s'" % (type_name, c, e))
TypeError: Parameters for type 'NoOptimizer' do not match: {}. Reason: 'Can't instantiate abstract class NoOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback'

======================================================================
ERROR: test_black_box_search.test_black_box_search_protocol
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/behavior_search/test/test_black_box_search.py", line 32, in test_black_box_search_protocol
    bs = BlackBoxSearch(ConstantBehavior(), NoOptimizer())
TypeError: Can't instantiate abstract class NoOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_black_box_search.test_save_black_box_search
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/behavior_search/test/test_black_box_search.py", line 47, in test_save_black_box_search
    bs = BlackBoxSearch(ConstantBehavior(), NoOptimizer())
TypeError: Can't instantiate abstract class NoOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_monte_carlo_rl.test_mc_rl
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/behavior_search/test/test_monte_carlo_rl.py", line 20, in test_mc_rl
    returns = ctrl.learn()
  File "/root/project/bolero-dev/learning/bolero/bolero/controller/controller.py", line 180, in learn
    feedbacks = self.episode(meta_parameter_keys, meta_parameters)
  File "/root/project/bolero-dev/learning/bolero/bolero/controller/controller.py", line 218, in episode
    meta_parameters)
  File "/root/project/bolero-dev/learning/bolero/bolero/controller/controller.py", line 280, in episode_with
    self.environment.step_action()
  File "/root/project/bolero-dev/learning/bolero/bolero/environment/openaigym.py", line 146, in step_action
    if self.step >= self.env.spec.timestep_limit:
AttributeError: 'EnvSpec' object has no attribute 'timestep_limit'

======================================================================
ERROR: test_controller.test_pickle_controller
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/controller/test/test_controller.py", line 127, in test_pickle_controller
    opt = Optimizer(initial_params=np.zeros(2))
TypeError: Can't instantiate abstract class REPSOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_environment.test_environments_follow_standard_protocol
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/environment/test/test_environment.py", line 39, in test_environments_follow_standard_protocol
    env.step_action()
  File "/root/project/bolero-dev/learning/bolero/bolero/environment/openaigym.py", line 146, in step_action
    if self.step >= self.env.spec.timestep_limit:
AttributeError: 'EnvSpec' object has no attribute 'timestep_limit'

======================================================================
ERROR: test_openaigym.test_discrete_input
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/environment/test/test_openaigym.py", line 24, in test_discrete_input
    env.step_action()
  File "/root/project/bolero-dev/learning/bolero/bolero/environment/openaigym.py", line 146, in step_action
    if self.step >= self.env.spec.timestep_limit:
AttributeError: 'EnvSpec' object has no attribute 'timestep_limit'

======================================================================
ERROR: test_openaigym.test_box_input
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/environment/test/test_openaigym.py", line 51, in test_box_input
    env.step_action()
  File "/root/project/bolero-dev/learning/bolero/bolero/environment/openaigym.py", line 146, in step_action
    if self.step >= self.env.spec.timestep_limit:
AttributeError: 'EnvSpec' object has no attribute 'timestep_limit'

======================================================================
ERROR: test_acmes.test_acmes_clip_samples
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_acmes.py", line 7, in test_acmes_clip_samples
    opt = ACMESOptimizer(n_train_max=20001)
TypeError: Can't instantiate abstract
 class ACMESOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_acmes.test_acmes_no_presamples
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_acmes.py", line 13, in test_acmes_no_presamples
    opt = ACMESOptimizer(n_pre_samples_per_update=0)
TypeError: Can't instantiate abstract class ACMESOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_baseline.test_no_optimizer
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_baseline.py", line 9, in test_no_optimizer
    opt = NoOptimizer(initial_params)
TypeError: Can't instantiate abstract class NoOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_baseline.test_random_optimizer
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_baseline.py", line 25, in test_random_optimizer
    opt = RandomOptimizer(initial_params)
TypeError: Can't instantiate abstract class RandomOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_cem.test_cem_no_initial_params
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_cem.py", line 10, in test_cem_no_initial_params
    opt = CEMOptimizer()
TypeError: Can't instantiate abstract class CEMOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_cem.test_cem_dimensions_mismatch
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_cem.py", line 17, in test_cem_dimensions_mismatch
    opt = CEMOptimizer(initial_params=np.zeros(5))
TypeError: Can't instantiate abstract class CEMOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_cem.test_cem_elite_fraction
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_cem.py", line 22, in test_cem_elite_fraction
    opt1 = CEMOptimizer(elite_frac=1.5)
TypeError: Can't instantiate abstract class CEMOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_cem.test_cem_get_best_params_mean
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_cem.py", line 31, in test_cem_get_best_params_mean
    opt = CEMOptimizer()
TypeError: Can't instantiate abstract class CEMOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_cem.test_cem_get_best_params_best
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_cem.py", line 41, in test_cem_get_best_params_best
    opt = CEMOptimizer()
TypeError: Can't instantiate abstract class CEMOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_rosenbrock.test_xnes
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_rosenbrock.py", line 40, in test_xnes
    opt = XNESOptimizer(x, random_state=0, log_to_stdout=False)
TypeError: Can't instantiate abstract class XNESOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_rosenbrock.test_reps
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_rosenbrock.py", line 48, in test_reps
    opt = REPSOptimizer(x, random_state=0, log_to_stdout=False)
TypeError: Can't instantiate abstract class REPSOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_rosenbrock.test_no
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_rosenbrock.py", line 56, in test_no
    opt = NoOptimizer(x)
TypeError: Can't instantiate abstract class NoOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_rosenbrock.test_random
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_rosenbrock.py", line 65, in test_random
    opt = RandomOptimizer(x, random_state=0)
TypeError: Can't instantiate abstract class RandomOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_rosenbrock.test_acmes
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/optimizer/test/test_rosenbrock.py", line 73, in test_acmes
    opt = ACMESOptimizer(x, random_state=0, log_to_stdout=F
alse)
TypeError: Can't instantiate abstract class ACMESOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback

======================================================================
ERROR: test_python_wrapper.test_load_behavior_search
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/project/bolero-dev/learning/bolero/bolero/wrapper/test/test_python_wrapper.py", line 55, in test_load_behavior_search
    bhs.init(n_inputs, n_outputs)
  File "bolero/wrapper/_wrapper.pyx", line 313, in bolero.wrapper._wrapper.CppBehaviorSearch.init
    self.thisptr.init(num_inputs, num_outputs, self.config_yaml)
RuntimeError: Python exception (TypeError): Parameters for type 'NoOptimizer' do not match: {'initial_params': [0]}. Reason: 'Can't instantiate abstract class NoOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback'
Traceback (most recent call last):
  File "/root/project/bolero-dev/learning/bolero/bolero/utils/module_loader.py", line 49, in behavior_search_from_yaml_string
    return from_dict(map_["BehaviorSearch"])
  File "/root/project/bolero-dev/learning/bolero/bolero/utils/module_loader.py", line 178, in from_dict
    result[k] = from_dict(v, name=k)
  File "/root/project/bolero-dev/learning/bolero/bolero/utils/module_loader.py", line 181, in from_dict
    return _from_dict(name, config)
  File "/root/project/bolero-dev/learning/bolero/bolero/utils/module_loader.py", line 247, in _from_dict
    "'%s'" % (type_name, c, e))
TypeError: Parameters for type 'NoOptimizer' do not match: {'initial_params': [0]}. Reason: 'Can't instantiate abstract class NoOptimizer with abstract methods get_batch_size, get_next_parameter_batch, set_batch_feedback'

----------------------------------------------------------------------
Ran 217 tests in 6.714s

FAILED (SKIP=1, errors=23)

Exited with code exit status 1

[AlexanderFabisch]

@maotto FYI, your docker works, but the tests currently fail.

[maotto]

Thanks for the info. I noticed it tests python2.7 - do we still want to support that?
I also don't find batch method definitions in the optimizer base class or with grep.
Plus, we shouldn't enforce special optimizer interfaces. Batch optimization related methods are nice to have, but for me, multi-objective optimization related methods would be as important. I understand the unit tests fail because the functionality isn't optional and I would fix it accordingly. Would you agree?

[AlexanderFabisch]

Thanks for the info. I noticed it tests python2.7 - do we still want to support that?

Not necessarily. We could base the docker image for CI on a newer version of Ubuntu, which has a more current Python 3 version as well probably.

I also don't find batch method definitions in the optimizer base class or with grep.

I'm a bit puzzled, too. I don't know where it comes from at the moment.

Plus, we shouldn't enforce special optimizer interfaces. Batch optimization related methods are nice to have, but for me, multi-objective optimization related methods would be as important. I understand the unit tests fail because the functionality isn't optional and I would fix it accordingly. Would you agree?

Yes, it should be optional.

[malter]

The batch computing is added on the branch distribution and not yet merged to the master. I thought the methods were added as optional extensions there.

[AlexanderFabisch]

Ah, OK #123 should be merged to distribution. Why is that the case? Shouldn't it be merged to master and distribution be rebased on master?