Fix INGO implementation

[not522]

Contributor Agreements

Please read the contributor agreements and if you agree, please click the checkbox below.

• [x] I agree to the contributor agreements.

[!TIP] Please follow the Quick TODO list to smoothly merge your PR.

Motivation

The implementation of ImplicitNaturalGradientSampler contains bugs. This PR fixes them and improves the sampling speed.

Description of the changes

• Fix direction of optimization
• Fix queue popping order
• Use optuna.search_space.IntersectionSearchSpace instead of optuna.samplers.IntersectionSearchSpace
• Remove unnecessary deepcopy

I have checked the implementation with the following code.

import optuna
import optunahub

def objective_L1_Ellipsoid(trial):
    DIMENSION = 100
    xs = [trial.suggest_float(f"x{n}", -1.0, 1.0) for n in range(DIMENSION)]
    return sum(10**(6*i/(DIMENSION-1))*abs(xs[i]) for i in range(DIMENSION))

mod = optunahub.load_local_module(
    package="samplers/implicit_natural_gradient",
    registry_root="./package",
    force_reload=True,
)

sampler = mod.ImplicitNaturalGradientSampler(seed=42)
study = optuna.create_study(sampler=sampler)
study.optimize(objective_L1_Ellipsoid, n_trials=10000)

The best value:

• main (with deepcopy=False): 1206069.1983931102
• This PR: 9595.199471317954

TODO List towards PR Merge

• [ ] Copy ./template/ to create your package
• [ ] Replace <COPYRIGHT HOLDER> in LICENSE of your package with your name
• [ ] Fill out README.md in your package
• [ ] Add import statements of your function or class names to be used in __init__.py
• [ ] Apply the formatter based on the tips in README.md
• [ ] Check whether your module works as intended based on the tips in README.md