optuna / optuna-dashboard

Real-time Web Dashboard for Optuna.
https://optuna-dashboard.readthedocs.io/en/latest/
Other
524 stars 88 forks source link

Show the best value on study detail page. #252

Open lyhue1991 opened 2 years ago

lyhue1991 commented 2 years ago

Description

When update to v0.7.0, the BestValue in the homepage is missed. But this information is very important for user to fast see the experiment's status. I think this may be a bug when

The homepage of v0.7.1

The homepage of v0.6.4

How to Reproduce

create an study like below using optuna and open optuna-dashboard to visualize.

import optuna
import datetime
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier

def objective(trial):
    data,target = make_classification(n_samples=1000,n_features=20,
            n_informative=12,n_redundant=4,n_repeated=0,n_classes=4,
            n_clusters_per_class=4)

    x_train, x_valid, y_train, y_valid = train_test_split(data, target)
    classes = list(set(target))

    clf = MLPClassifier(
        hidden_layer_sizes=tuple(
            [trial.suggest_int("n_units_l{}".format(i), 32, 64) for i in range(5)]
        ),
        learning_rate_init=trial.suggest_float("lr_init", 1e-5, 1e-1, log=True),
    )

    for step in range(200):
        clf.partial_fit(x_train, y_train, classes=classes)
        value = clf.score(x_valid, y_valid)

        # Report intermediate objective value.
        trial.report(value, step)

        # Handle pruning based on the intermediate value.
        if trial.should_prune():
            raise optuna.TrialPruned()

    return value

storage_name = "sqlite:///optuna.db"
nowtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
study = optuna.create_study(direction="maximize", pruner=optuna.pruners.MedianPruner(),
                            study_name="mlp", storage=storage_name, load_if_exists=True)
study.optimize(objective, n_trials=200, timeout=3600)

Python version

3.7

Optuna version

Output of $ python -c 'import optuna; print(optuna.__version__)'

optuna-dashboard version or git revision

0.7.1

Web browser

chrome

c-bata commented 2 years ago

Thank you for opening an issue. I deliberately removed the "Best Value" column due to the performance reasons.

Therefore, you need to sort trials by objective value in TrialTable in order to check the best value. It is a bit cumbersome, so I will try to think of a way to see "Best Value" easily as before. Sorry for the inconvenience.