automl / neps

Neural Pipeline Search (NePS): Helps deep learning experts find the best neural pipeline.
https://automl.github.io/neps/
Apache License 2.0
44 stars 11 forks source link

[UX] Show error and traceback when something goes wrong #62

Open eddiebergman opened 3 months ago

eddiebergman commented 3 months ago

While testing some new things, I get this error and it's got no information useful for understanding what went wrong.

/home/skantify/code/neps/neps_examples/efficiency/multi_fidelity.py:81: in <module>
    neps.run(
/home/skantify/code/neps/neps/api.py:273: in run
    metahyper_run(
/home/skantify/code/neps/neps/metahyper/api.py:586: in metahyper_run
    post_evaluation_hook(
/home/skantify/code/neps/neps/api.py:39: in _post_evaluation_hook
    loss = get_loss(result, loss_value_on_error, ignore_errors)

    def get_loss(
        result: str | dict | float,
        loss_value_on_error: float | None = None,
        ignore_errors: bool = False,
    ) -> float | Any:
        if result == "error":
            if ignore_errors:
                return "error"
            elif loss_value_on_error is None:
>               raise ValueError(
                    "An error happened during the execution of your run_pipeline function."
                    " You have three options: 1. If the error is expected and corresponds to"
                    " a loss value in your application (e.g., 0% accuracy), you can set"
                    " loss_value_on_error to some float. 2. If sometimes your pipeline"
                    " crashes randomly, you can set ignore_errors=True. 3. Fix your error."
                )
E               ValueError: An error happened during the execution of your run_pipeline function. You have three options: 1. If the error is expected and corresponds to a loss value in your application (e.g., 0% accuracy), you can set loss_value_on_error to some float. 2. If sometimes your pipeline crashes randomly, you can set ignore_errors=True. 3. Fix your error.

It did show something in the logs which is nice but I feel like these errors should get bubbled all the way up.