automl / neps

Neural Pipeline Search (NePS): Helps deep learning experts find the best neural pipeline.
https://automl.github.io/neps/
Apache License 2.0
39 stars 11 forks source link

Error caused by post_run_summary argument if using hyperband as a searcher #104

Closed danrgll closed 4 weeks ago

danrgll commented 1 month ago

Example to reproduce the error:

def run_pipeline(**config):
    epochs = config["epochs"]
    optimizer = config["optimizer"]
    eval_score = np.random.random(1)
    return {"loss": eval_score}

pipeline_space = dict(
    epochs=neps.IntegerParameter(lower=1, upper=10, is_fidelity=True),
    optimizer=neps.CategoricalParameter(choices=["sgd", "adam"])
)

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    neps.run(run_pipeline=run_pipeline, pipeline_space=pipeline_space,
             root_directory="results", max_evaluations_total=20,
             overwrite_working_directory=True, post_run_summary=True)

Error message:

INFO:neps:Start evaluating config 14_2 INFO:neps:Finished evaluating config 14_2 INFO:neps:Finished evaluating config 14_2 INFO:neps:Maximum total evaluations is reached, shutting down Traceback (most recent call last): File "/Users/daniel/PycharmProjects/neps/wrapper_test.py", line 31, in neps.run(run_pipeline=run_pipeline, pipeline_space=pipeline_space, File "/Users/daniel/PycharmProjects/neps/neps/api.py", line 353, in run post_run_csv(root_directory) File "/Users/daniel/PycharmProjects/neps/neps/status/status.py", line 344, in post_run_csv df_config_data, df_run_data = _get_dataframes_from_summary( File "/Users/daniel/PycharmProjects/neps/neps/status/status.py", line 205, in _get_dataframes_from_summary summary = get_summary_dict(root_directory=root_directory, add_details=True) File "/Users/daniel/PycharmProjects/neps/neps/status/status.py", line 39, in get_summary_dict shared_state.update_from_disk() File "/Users/daniel/PycharmProjects/neps/neps/runtime.py", line 626, in update_from_disk previous_report = self.evaluated_trials[previous_config_id] KeyError: '3_0'

danrgll commented 1 month ago

The error message looks similar to error message of #103 but caused differently