Exploring the generated sims

[berceanu]

Let's say the user runs an intensive Grid/Line sampling campaign with hundreds of runs. How does one explore the generate sims afterwards? There are folders evaluations/sim<NNNN>, however there doesn't seem to be a mapping from folder name to pair of parameter values used for running the simulation in that particular folder.

A concrete example. Let's say we do a Grid Sampling on GPU via fbpic and run 100 sims. I would then like to know in which folder I can find the output of the simulation with (a0=2, tau=24, w0=12). Or all the folders that contain sims with a0>3, and so on.

For example, signac implements this, see eg exploring data

I guess a good starting point would be the ExplorationDiagnostics class methods such as get_evaluation_dir_path?

[shuds13]

libensemble does not currently have such naming options built in (though its possible to do it manually within a sim function).

I think we could support something, but one consideration would be if inputs were floats, you would want to print only to some significant figures and could get clashes. Then maybe prefixing with the sim_id would be desirable to prevent that.

[AngelFP]

Hi @berceanu. As you pointed out, the ExplorationDiagnostics class is what you need. It contains the history of the exploration, the path to each evaluation, plotting methods and other stuff. Did you have any trouble using it?

A simple example to get the path to all simulations with a0>3:

diags = ExplorationDiagnostics("path/to/exploration")
best_evals = diags.history.query("a0 > 3")
for trial_index in best_evals["trial_index"]:
    print(diags.get_evaluation_dir_path(trial_index))

[berceanu]

I see, so one can use the pandas API basically. Thanks, I think we can close this and I will reopen if I encounter any further issues.