Wrong error raised by BTBSession on too many errors

[csala]

• BTB version > 0.3.5

Description

There is an error in the way the remaining options for the BTBSession are evaluate which makes the session crash with a misleading exception when it has actually run out of options.

What I Did

The erro can be provoked by using a single scorer that raises an exception. After the max_errors is reached, the expected StopTuning exception is not being raised, but rather a misleading ValueError:

In [1]: tunables = { 
   ...:     'a': { 
   ...:         'a': {'type': 'int', 'default': 0, 'range': [0, 100]}, 
   ...:     } 
   ...: } 
   ...:  
   ...: def scorer(name, hyperparameters): 
   ...:     raise Exception() 
   ...:  
   ...: from btb.session import BTBSession 
   ...:  
   ...: session = BTBSession(tunables, scorer, verbose=True) 
   ...:  
   ...: session.run(5) 
   ...:
ValueError                                Traceback (most recent call last)
<ipython-input-1-34ba33972a06> in <module>
     12 session = BTBSession(tunables, scorer, verbose=True)
     13 
---> 14 session.run(5)

~/Projects/MIT/BTB/btb/session.py in run(self, iterations)
    296         for _ in iterator:
    297             self.iterations += 1
--> 298             tunable_name, config = self.propose()
    299 
    300             try:

~/Projects/MIT/BTB/btb/session.py in propose(self)
    199 
    200         else:
--> 201             tunable_name = self._selector.select(self._normalized_scores)
    202             tuner = self._tuners[tunable_name]
    203             try:

~/Projects/MIT/BTB/btb/selection/selector.py in select(self, choice_scores)
     63             choice_rewards[choice] = self.compute_rewards(scores)
     64 
---> 65         return self.bandit(choice_rewards)

~/Projects/MIT/BTB/btb/selection/ucb1.py in bandit(self, choice_rewards)
     46             return average_reward + error
     47 
---> 48         return max(self._shuffle(choice_rewards), key=ucb1)

ValueError: max() arg is an empty sequence