search

optuna / optuna-dashboard

Real-time Web Dashboard for Optuna.
https://optuna-dashboard.readthedocs.io/en/latest/
Other
534 stars 90 forks source link

Contour plot visualization issue with categorical parameters in Optuna Dashboard #978

Closed chekh closed 3 weeks ago

chekh commented 1 month ago

Description

There is an issue with the visualization of contour plots in Optuna Dashboard when using categorical parameters. The problem occurs when trying to plot a contour graph with a categorical parameter on either the X or Y axis. The dashboard page turns completely white, and no content is displayed, although data is successfully loaded into the browser.

The issue seems to stem from an error in the JavaScript bundle.js script, which prevents the proper rendering of the page. This behavior is reproducible by running the provided Python script, launching the Optuna Dashboard, and navigating to the "Analytics" page to select categorical parameters for the contour plot.

This problem affects users who rely on Optuna Dashboard for visualizing the results of optimization studies, especially when using categorical parameters in their trials.

Chrome console log: 

bundle.js:102 TypeError: Cannot set properties of undefined (setting '6')
    at bundle.js:75:365716
    at Array.forEach (<anonymous>)
    at hM (bundle.js:75:365501)
    at bundle.js:75:363420
    at rl (bundle.js:2:7023060)
    at _c (bundle.js:2:7043005)
    at bundle.js:2:7041782
    at xc (bundle.js:2:7041842)
    at lc (bundle.js:2:7035658)
    at Vi (bundle.js:2:6976686)
    at bundle.js:2:7033086
console.error @ bundle.js:102
fs @ bundle.js:2
n.callback @ bundle.js:2
ja @ bundle.js:2
Tl @ bundle.js:2
bl @ bundle.js:2
yl @ bundle.js:2
(anonymous) @ bundle.js:2
xc @ bundle.js:2
lc @ bundle.js:2
Vi @ bundle.js:2
(anonymous) @ bundle.js:2
bundle.js:75 Uncaught TypeError: Cannot set properties of undefined (setting '6')
    at bundle.js:75:365716
    at Array.forEach (<anonymous>)
    at hM (bundle.js:75:365501)
    at bundle.js:75:363420
    at rl (bundle.js:2:7023060)
    at _c (bundle.js:2:7043005)
    at bundle.js:2:7041782
    at xc (bundle.js:2:7041842)
    at lc (bundle.js:2:7035658)
    at Vi (bundle.js:2:6976686)
    at bundle.js:2:7033086
(anonymous) @ bundle.js:75
hM @ bundle.js:75
(anonymous) @ bundle.js:75
rl @ bundle.js:2
_c @ bundle.js:2
(anonymous) @ bundle.js:2
xc @ bundle.js:2
lc @ bundle.js:2
Vi @ bundle.js:2
(anonymous) @ bundle.js:2

How to Reproduce

  1. Optuna's Dashboard "Analitics" page works correctly with various types of parameters, including categorical ones. The issue appears when visualizing contour plots with categorical parameters.

  2. Run the attached code: Use the provided Python script to create and optimize a study using Optuna, then launch the Optuna Dashboard.

  3. Open the "Analytics" page, then select any categorical parameter for either axis in the contour plot. The problem occurs when selecting a categorical parameter.

  4. An error occurs: When selecting a categorical parameter for the X or Y axis in the contour plot, the entire dashboard page turns white, leaving only a blank screen. Data is successfully loaded into the browser, but an error occurs in the bundle.js script, causing the page to fail rendering.

Test code to reproduce bug:

import random
import optuna
from optuna.storages import RDBStorage
import logging
from optuna_dashboard import run_server  # Updated import

# Set logging level to debug
logging.basicConfig(level=logging.DEBUG)

# Use an in-memory SQLite database
storage = RDBStorage("sqlite:///:memory:")

# Variable for the experiment name (can be changed if necessary)
study_name = "example_study_v_21"  # Experiment name

# Variable to specify the number of parameters
num_int_params = 3
num_float_params = 3
num_list_str_params = 3
num_list_int_params = 3
num_parameters = num_int_params + num_float_params + num_list_str_params + num_list_int_params

# Function for generating random trial values
def random_value() -> float:
    return random.random()

# Define the objective function
def objective(trial: optuna.Trial) -> float:
    result = 0

    # Log the beginning of a trial
    logging.debug(f"Starting trial {trial.number}")

    # Optimize float parameters
    for i in range(num_float_params):
        x = trial.suggest_float(f'float_param_{i}', -10.0, 10.0)
        logging.debug(f"Float parameter {i}: {x}")
        result += random_value()

    # Optimize int parameters
    for i in range(num_int_params):
        x = trial.suggest_int(f'int_param_{i}', 1, 100)
        logging.debug(f"Int parameter {i}: {x}")
        result += random_value()

    # === Unique: Using suggest_categorical for string parameters ===
    # Choosing this parameter on contour plot destroy the dashboard
    str_param_1 = trial.suggest_categorical(f'str_param_1', ["option1", "option2", "option3"])
    logging.debug(f"String parameter (suggest_categorical): {str_param_1}")
    result += random_value()

    # === Unique: Using numeric index to represent string options ===
    options = ["low", "medium", "high"]
    index = trial.suggest_int(f'str_param_2_index', 0, len(options) - 1)
    str_param_2 = options[index]
    logging.debug(f"String parameter (numeric index): {str_param_2}")
    result += random_value()

    # === Unique: Combined categorical parameters with other types ===
    # Choosing this parameter on contour plot destroy the dashboard
    category = trial.suggest_categorical(f'category_param', ["catA", "catB"])
    if category == "catA":
        param_a = trial.suggest_float("param_a", 0.0, 1.0)
        logging.debug(f"Category catA selected, param_a: {param_a}")
        result += random_value()
    elif category == "catB":
        param_b = trial.suggest_int("param_b", 1, 10)
        logging.debug(f"Category catB selected, param_b: {param_b}")
        result += random_value()

    # === Unique: Using string to number transformation ===
    # Choosing this parameter on contour plot destroy the dashboard
    options_map = {"low": 1, "medium": 2, "high": 3}
    str_param_4 = trial.suggest_categorical(f'str_param_4_transform', list(options_map.keys()))
    transformed_value = options_map[str_param_4]
    logging.debug(f"String parameter (transformed to number): {str_param_4}, value: {transformed_value}")
    result += random_value()

    # === Unique: Using integer numbers to represent categories ===
    # Choosing this parameter on contour plot destroy the dashboard
    options_list = [1, 3, 5, 7]
    int_category_1 = trial.suggest_categorical(f'int_category_1', options_list)
    logging.debug(f"Integer parameter from category: {options_list}, value: {int_category_1}")
    result += random_value()

    # Set user attributes for each trial
    trial.set_user_attr("custom_info", f"Trial with index {trial.number}")
    logging.debug(f"Final result for trial {trial.number}: {result}")

    return result

# Create a new study or load an existing one
study = optuna.create_study(study_name=study_name, storage=storage, load_if_exists=True)

# Add user attributes for the study
study.set_user_attr("study_info", "This study includes various types of parameters")

# Run optimization
study.optimize(objective, n_trials=10)

# Start the Optuna dashboard server after the trials
run_server(storage, '127.0.0.1', 8080)

Python version

3.11

Optuna version

4.0.0

optuna-dashboard version or git revision

0.16.2

Web browser

Google Chrome, Yandex, Safari

berombau commented 4 weeks ago

👍 I have the same issue

xianglun918 commented 3 weeks ago

I have the same issue. Seems to have something to do with the contour graph rendering.

c-bata commented 3 weeks ago

Thank you for your report. @porink0424 is working on fixing this bug 🙏

c-bata commented 3 weeks ago

@chekh @berombau @xianglun918 @ZWL-S @thomasroodnl Thank you again for reporting this bug. It has been fixed in #988 and is now available in the v0.17.0 release! https://github.com/optuna/optuna-dashboard/releases/tag/v0.17.0 https://pypi.org/project/optuna-dashboard/0.17.0/