Support multi-objective study for `from_optuna`

[nzw0301]

Hi, thank you for introducing the Optuna integration by #215!

I suppose the current implementation does not support study whose objective function returns multiple objective values. More concretely, the following code:

import optuna
import hiplot as hip

def objective(trial: "optuna.trial.Trial") -> float:
    x = trial.suggest_float("x", -1, 1)
    y = trial.suggest_float("y", -1, 1)    

    return x ** 2, y

study = optuna.create_study(directions=["minimize"]*2)
study.optimize(objective, n_trials=3)

xp = hip.Experiment.from_optuna(study)

The error message is as follows.

RuntimeError                              Traceback (most recent call last)
/var/folders/n3/7_7r1yrx6jsc_0780bvg02lr0000gn/T/ipykernel_82314/3836556673.py in <module>
      2 study.optimize(objective, n_trials=3)
      3 
----> 4 xp = hip.Experiment.from_optuna(study)

~/Documents/hiplot/hiplot/experiment.py in from_optuna(study)
    519         hyper_opt_data = []
    520         for each_trial in study.trials:
--> 521             trial_params = {}
    522             trial_params["value"] = each_trial.value # name = value, as it could be RMSE / accuracy, or any value that the user selects for tuning
    523             trial_params["uid"] = each_trial.number

/opt/homebrew/Caskroom/miniconda/base/envs/optuna39/lib/python3.9/site-packages/optuna/trial/_frozen.py in value(self)
    389         if self._values is not None:
    390             if len(self._values) > 1:
--> 391                 raise RuntimeError(
    392                     "This attribute is not available during multi-objective optimization."
    393                 )

RuntimeError: This attribute is not available during multi-objective optimization.

Dependencies:

• Optuna: 2.10.0
• hiplot: https://github.com/facebookresearch/hiplot/tree/79b3d52a6842d6ba12f0a544e27a444562a486df

---

To clarify it in the documentation, this PR mentions the supported study.

By the way, to access the objective values of either single and multi-objective, we can use tiral.values that contains retuned value by the objective function of Optuna as list.

[danthe3rd]

Thanks for noticing and reacting :) Happy to accept this - but maybe the correct fix would be to indeed display the multiple objective values. Let me know if that's something you want to add to this PR.

cc @GoldenCorgi

[nzw0301]

Indeed, after sending this PR, I realised that we can support the multi-objective case according to the definition of hiplot.datapoint. I'll revert the docs change and introduce a logic for the multi-objective case.

[nzw0301]

hip.Experiment.from_optuna(study) can deal with the multi-objective case as follows by 283d624 .

print(xp._asdict())
{'parameters_definition': {},
 'colormap': 'interpolateTurbo',
 'colorby': None,
 'weightcolumn': None,
 'display_data': {},
 'datapoints': [{'uid': '0',
   'values': {'value_0': 0.9962397033748125,
    'value_1': 0.29712304597312356,
    'x': 0.9981180808776147,
    'y': 0.29712304597312356},
   'from_uid': None},
  {'uid': '1',
   'values': {'value_0': 0.9955093535069702,
    'value_1': 0.9807527166484111,
    'x': -0.9977521503394369,
    'y': 0.9807527166484111},
   'from_uid': None},
  {'uid': '2',
   'values': {'value_0': 0.8412255761276068,
    'value_1': -0.3708578580795301,
    'x': 0.917183501883678,
    'y': -0.3708578580795301},
   'from_uid': None}]}

[danthe3rd]

Thanks for updating! Can we also add a test for the multi-objectives case - similar to this one ?

[nzw0301]

Definitely! b063c53

[GoldenCorgi]

Thanks so much for pointing this out! And also for correcting the code for the Optuna integration :)

The fix looks good on my end, do let me know if there's other Optuna edge cases that I might have missed!

[danthe3rd]

CI is all green - merging I'm pushing version 0.1.31 with this fix - should be available within ~1h if everything goes well. Thanks both of you :)

[nzw0301]

Thank you for giving feedback so quickly!

I also proposed adding an example code using hiplot in optuna example directory as https://github.com/optuna/optuna-examples/issues/65.

Cheers,