High dimensional search space limit

[23pointsNorth]

Describe the bug When building large search spaces over dimensionality of 32, a numpy limit is reached.

Code to reproduce the behavior

import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer

def parabola_function(para):
    loss = para["x0"] * para["x0"]
    return -loss

search_dim = 50

search_space = {
    f"x{i}": np.arange(-10, 10, 1) for i in range(search_dim)
}

print(search_space)
opt = RandomSearchOptimizer(search_space)
opt.search(parabola_function, n_iter=10)

Error message from command line

    opt.search(parabola_function, n_iter=10)
  File "/home/daniel/.local/lib/python3.8/site-packages/gradient_free_optimizers/search.py", line 191, in search
    self.init_positions = init.set_pos(self.initialize)
  File "/home/daniel/.local/lib/python3.8/site-packages/gradient_free_optimizers/init_positions.py", line 20, in set_pos
    positions = self._init_grid_search(initialize["grid"])
  File "/home/daniel/.local/lib/python3.8/site-packages/gradient_free_optimizers/init_positions.py", line 85, in _init_grid_search
    pos_mesh = np.array(np.meshgrid(*positions))
  File "<__array_function__ internals>", line 5, in meshgrid
  File "/home/daniel/.local/lib/python3.8/site-packages/numpy/lib/function_base.py", line 4360, in meshgrid
    output = [np.asanyarray(x).reshape(s0[:i] + (-1,) + s0[i + 1:])
  File "/home/daniel/.local/lib/python3.8/site-packages/numpy/lib/function_base.py", line 4360, in <listcomp>
    output = [np.asanyarray(x).reshape(s0[:i] + (-1,) + s0[i + 1:])
ValueError: maximum supported dimension for an ndarray is 32, found 50

System information:

• OS Platform and Distribution Ubuntu20.04
• Python version 3.8
• Gradient-Free-Optimizers version 0.4.2 (+ main branch tested)

Additional context

[SimonBlanke]

Hello @23pointsNorth,

nice find! I did not know this limitation of numpy.

However it seems, that this issue is related or a duplicate of issue SimonBlanke/Gradient-Free-Optimizers#1, because of this line:

pos_mesh = np.array(np.meshgrid(*positions))

The bug is caused by the calculation of the initial positions as vertices and grids in the search space. I initialized with random and the code runs fine:

import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer

def parabola_function(para):
    loss = para["x0"] * para["x0"]
    return -loss

search_dim = 50
search_space = {f"x{i}": np.arange(-10, 10, 1) for i in range(search_dim)}

print(search_space)
opt = RandomSearchOptimizer(search_space, initialize={"random": 4})
opt.search(parabola_function, n_iter=10)

I will close this issue for now. If the numpy-limit becomes a problem for GFO we can reopen it.

Thanks for the help! :-)

[23pointsNorth]

Just wanted to confirm that your suggestion fixed the issue. Thanks for the quick response!

[SimonBlanke]

@23pointsNorth

very nice! I just rolled out v1.0.5, which has a fix/workaround for the "vertices"- and "grid"-initializations.