can the ask function be altered to return top n suggestions instead of just one?

[vaish1]

Hi, I want to parallelize the optimization procedure by evaluating not just the one best suggestion but checking a few top suggestions that the ask function returns. Is there a way to do this with TreeParzen? Can I modify the ask function in API.jl to return more than one sample? Thanks in advance!

[BojieSheng]

Hi Vaish1, please check below code if it is what you want.

using TreeParzen
import TreeParzen: Trials, Types

function askmany(space::Types.SPACE_TYPE,
    trials::Vector{Trials.Trial},
    config::Config;
    n_suggestions::Int=1,
)::Vector{Trials.Trial}
    trial_list = Trials.Trial[]
    while length(trial_list) < n_suggestions
        buf_trial = ask(space, trials, config)
        if !(buf_trial in trial_list)
            push!(trial_list, buf_trial)
        end
    end

    return trial_list

end

config = TreeParzen.Config()
trialhist = Trials.Trial[]
space = Dict(:x => HP.Uniform(:x, -5., 5.))
TreeParzen.Graph.checkspace(space)

# Ask for a new trial
# Test askmany function
for i in 1:10
    # suggest 3 new hyperparameters to try
    trial_list = askmany(space,
        trialhist,
        config,
        n_suggestions=3,
    )
    @show trial_list
    # evaluate each hyperparameter and update the trial history
    for trial in trial_list
        objective_value = trial.hyperparams[:x] ^ 2
        tell!(trialhist, trial, objective_value)
    end
end

@show provide_recommendation(trialhist)

[vaish1]

Thanks BojieSheng, this helps a lot! :)