Epistimio / orion

Asynchronous Distributed Hyperparameter Optimization.
https://orion.readthedocs.io
Other
287 stars 52 forks source link

[WIP] Add new analysis method for symbolic explanation #1118

Open bouthilx opened 1 year ago

bouthilx commented 1 year ago

Add new analysis method for symbolic explanation, coming from https://openreview.net/forum?id=JQwAc91sg_x

This is still work in progress. Need to validate how sensitive the method is to the choice of population size, number of generations and parsimony coefficient.

The visualization reuses a lot of the code of the partial dependencies as a boilerplate. I would need to refactor the code to be able to reuse it instead of duplicating it.

Needs to add unit-tests.

It can be tested manually using the following snippet of code:

from orion.analysis.symbolic_explanation import SymbolicRegressorParams
from orion.client import workon

def function_a(x):
    return 2 * x**2

def function_b(x, y):
    return 2 * x**2 + x * y + 3 * y**2

def function_c(x, y, z):
    return 1.5 * x**2 * y + z**2 / 2

def function_d(x, y, z, w):
    return 2 * x**2 + 3 * y**2 + z**2 + w**2

for function, space in [
    (function_a, {"x": "uniform(-5, 5)"}),
    (function_b, {"x": "uniform(-5, 5)", "y": "uniform(-5,5)"}),
    (
        function_c,
        {"x": "uniform(-5, 5)", "y": "uniform(-5,5)", "z": "uniform(-5, 5)"},
    ),
    (
        function_d,
        {
            "x": "uniform(-5, 5)",
            "y": "uniform(-5,5)",
            "z": "uniform(-5, 5)",
            "w": "uniform(-5, 5)",
        },
    ),
]:
    experiment = workon(function, space, name=f"{len(space)}-dims", max_trials=200)
    fig = experiment.plot.symbolic_explanation(
        n_samples=200,
        symbolic_regressor_params=SymbolicRegressorParams(
            population_size=10000,
            generations=20,
            parsimony_coefficient=0.1,
        ),
    )
    fig.write_html(
        f"test_symbolic_pd_ndims_{len(experiment.space)}.html",
        include_mathjax="cdn",
    )