AIworx-Labs / chocolate

A fully decentralized hyperparameter optimization framework
http://chocolate.readthedocs.io
BSD 3-Clause "New" or "Revised" License
121 stars 41 forks source link

MemoryError using Random Sampler #31

Open TillBeemelmanns opened 5 years ago

TillBeemelmanns commented 5 years ago
import chocolate as choco

def get_op_space(i):
    return  {
                  "{}_posterize".format(i)        :  {"{}_l_posterize".format(i):          choco.quantized_uniform(0, 11, 1),  "{}_p_posterize".format(i):          choco.quantized_uniform(0, 1, 0.1)},
                  "{}_solarize".format(i)         :  {"{}_l_solarize".format(i):           choco.quantized_uniform(0, 11, 1),  "{}_p_solarize".format(i):           choco.quantized_uniform(0, 1, 0.1)},
                  "{}_adjust_brightness".format(i):  {"{}_l_adjust_brightness".format(i):  choco.quantized_uniform(0, 11, 1),  "{}_p_adjust_brightness".format(i):  choco.quantized_uniform(0, 1, 0.1)}
             }

space = {
    "sp1_op1": get_op_space("sp1_op1"),
    "sp1_op2": get_op_space("sp1_op2"),
    "sp2_op1": get_op_space("sp2_op1"),
    "sp2_op2": get_op_space("sp2_op2")
}

conn = choco.DataFrameConnection(from_file=None)

sampler = choco.Random(conn, space, random_state=None)

token, params = sampler.next()
Traceback (most recent call last):
  File "/src/search/test.py", line 22, in <module>
    token, params = sampler.next()
  File "/venv/lib/python3.5/site-packages/chocolate/base.py", line 159, in next
    return self._next()
  File "venv/lib/python3.5/site-packages/chocolate/sample/random.py", line 78, in _next
    choices = sorted(set(range(l)) - set(drawn))
MemoryError

I am trying to search within augmentation cascade search space as like in Googles Autoaugment. Each subpolicy (sp) consists of two operations with where each operation has a level (l) and probability (p). When I try to sample from this space I am encountering a MemoryError.