[GENERAL SUPPORT]: Plot observed pareto frontier alongside remaining Arms

[Jekannadar]

Question

For a multi-objective optimization with two output dimensions, is there an easy way, using the Service API, to plot not just the observed pareto optimal points, but to also visualize the remaining observed values of the arms?

Or do I need to modify the underlying AxPlotConfig to achieve this?

I envision something similar to what frameworks like Optuna can provide (see attached image).

Please provide any relevant code snippet if applicable.

observed_pareto_frontier = get_observed_pareto_frontiers(
        experiment=ax_client.experiment,
        data=ax_client.experiment.fetch_data(),
    )[0]
# Is there a simple way to add the points not on the observed pareto frontier in a different color?
render(plot_pareto_frontier(observed_pareto_frontier, CI_level=0.90))

Code of Conduct

• [X] I agree to follow this Ax's Code of Conduct

[mpolson64]

@Jekannadar yes, we have a number of functions for plotting all observations along with the pareto frontier (demarcated as a line through the points on the frontier) in Ax. The most straightforward way to do this is by using a helper function in our report_utils module:

from ax.service.utils.report_utils import _pareto_frontier_scatter_2d_plotly

_pareto_frontier_scatter_2d_plotly(experiment=ax_client.experiment)

Note that the plot is automatically zoomed in on the region of interest, so if you would like to see all of your observations you may need to double click on the plotly figure to zoom out.

As an aside, this function is rather hidden and it's admittedly not the best user experience wrt. ease of use or discoverability. The team is working on a rather large revamp of our plotting and general "analysis" code and hope to share some new features with the community in the coming months. We're really excited to be able to bring some usability improvements to this part of the Ax experience and to make it even easier for people to use state of the art Bayesian optimization methods in their projects.

Closing this issue for now, but feel free to reopen if you have any follow up questions.

[mpolson64]

For reference, here is a plot produced using the above snippet and the Branin-Currin multiobjective problem present in this tutorial: https://ax.dev/versions/0.3.5/tutorials/multiobjective_optimization.html