CMA bug for ill-scaled search space persists with `with_margin=True`

[contramundum53]

Expected behavior

Optimization runs efficiently

Environment

N/A

Error messages, stack traces, or logs

N/A

Steps to reproduce

import optuna

def objective(trial):
    x = trial.suggest_float("x", 0.5, 2.5)
    y = trial.suggest_int("y", 10, 200)
    return (x - 1.7) ** 2 + (y - 60) ** 2

if __name__ == "__main__":
    cmaes_sampler = optuna.samplers.CmaEsSampler()
    cmaes_sampler_wm = optuna.samplers.CmaEsSampler(with_margin=True)

    study = optuna.create_study(sampler=cmaes_sampler)
    study_wm = optuna.create_study(sampler=cmaes_sampler_wm)

    N = 1000

    study.optimize(objective, n_trials=N)
    study_wm.optimize(objective, n_trials=N)

    print("cmaes_sampler:")
    print(f"Best value: {study.best_value} (params: {study.best_params}, Trial number is {study.best_trial.number})")
    print("cmaes_sampler_with_margin:")
    print(f"Best value: {study_wm.best_value} (params: {study_wm.best_params}, Trial number is {study_wm.best_trial.number})")

Additional context (optional)

This bug should have been fixed in #4443, but it seems to persist with with_margin=True option.

[contramundum53]

Fixed in #4661 .