optuna / optuna-dashboard

Real-time Web Dashboard for Optuna.
https://optuna-dashboard.readthedocs.io/en/latest/
Other
515 stars 86 forks source link

Slice plot does not support boolean parameter values #977

Open cartisan opened 3 weeks ago

cartisan commented 3 weeks ago

Description

Observed behavior: When I select a categorical parameter that contains boolean values in the slice plot (first plot shown in the analytics page) then both values on the x-axis are represented as [Object object]. image

Expected behavior: The plot shows the actual values, True and False on the x-axis.

Observations: This seems a problem with optuna-dashbord and not optuna, when I create a slice plot in optuna (code below) I see the values represented correctly in the plotly graph. image

I'm not familiar with React, but the problem seems to be here:

  const feasibleValues = feasibleTrials.map(
    (t) => selectedParamTarget.getTargetValue(t) as number
  )

where the code assumes the values of the x-axis are numbers, which is not the case for boolean hyperspace parameters. I would guess that this also breaks for string values of categorical parameters.

Code to reproduce optuna plot

import optuna
from plotly.io import show
def objective(trial):
    x = trial.suggest_float("x", -100, 100)
    y = trial.suggest_categorical("y", [True, False])
    if y:
        return x**2
    else:
        return x*2
sampler = optuna.samplers.TPESampler(seed=10)
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=10)
fig = optuna.visualization.plot_slice(study, params=["x", "y"])
show(fig)

How to Reproduce

  1. Optuna's objective function is not relevant here, but search space needs to contain a categorical parameter that contains boolean values.
  2. Run optuna-dashboard.
  3. Open analytics page, then select a boolean parameter in the slice plot.
  4. An error occurs.

Python version

3.11

Optuna version

3.6.1

optuna-dashboard version or git revision

0.16.2

Web browser

Google Chrome

wassname commented 2 weeks ago

Ah I think I found it. It seems that the way categorical as stored has changed.

So

 (c) => c?.toString() ?? "null"

to

 (c) => c?.value?.toString() ?? "null"

everywhere it occurs

image

But I haven't run unit tests on it to check if this fix causes unintended problems, so I can't make it a PR yet

Spider-netizen commented 1 week ago

@wassname Thanks for this. Can you tell what the last working version would be?

wassname commented 1 week ago

I'm not sure sorry, I've only recently starting using it, so I don't know if it was ever working. But that's normal, it's a active volunteer project.