search

automl / NASLib

NASLib is a Neural Architecture Search (NAS) library for facilitating NAS research for the community by providing interfaces to several state-of-the-art NAS search spaces and optimizers.
Apache License 2.0
522 stars 117 forks source link

Storing of models or architectures in optimizers #134

Open jr2021 opened 2 years ago

jr2021 commented 2 years ago

In the get_candidates function of the zerocost branch optimizers Bananas and Npenas there is a discrepancy between how candidates in the next_batch are stored.

If the acquisition function is being optimized via "random_sampling", then model is being stored:

...
candidates.append(model)

Otherwise, if it is being optimized via "mutation", then model.arch is being stored:

...
    candidate = arch
candidates.append(candidate)

However, the function get_best_candidates (which is called directly after get_new_candidates) treats candidates as a list of models:

values = [acq_fn(model.arch, [{'zero_cost_scores' : model.zc_scores}]) for model in candidates]

Does this imply that the optimization of the acquisition function via "mutation" is not used in the main loop of either Bananas or Npenas? If so, how and when should this option be used?

Neonkraft commented 2 years ago

We used only random sampling for the optimization in the paper. You can change the mutation code to comply with the RS acquisition function.