handle error in `run_cv()`

[kallewesterling]

handle the case where an Exception is raised in cv_results = cross_validate(), which enters the except block, but we still returns cv_results which does not exist at that point.

def run_cv(X, y, cv, model, metrics, n_jobs, logger):
    model_name = get_model_name(model)

    # The metrics we want to use for cross-validation
    scorers = {metric.__name__: make_scorer(metric) for metric in metrics}

    logger.info(f"Cross-validating {model_name}...")
    logger.info(f"Parameters: {model.named_steps['model'].get_params()}")

    try:
        cv_results = cross_validate(
            model,
            X,
            y,
            cv=cv,
            scoring=scorers,
            n_jobs=n_jobs,
            return_estimator=True,
            return_indices=True,
        )

    except Exception as e:
        logger.error(f"Failed to cross-validate {model_name}")
        logger.error(e)

    return cv_results

[kallewesterling]

Hey @bryanlimy -- do you have a status update on this one?

[bryanlimy]

I add cv_results=None in https://github.com/alan-turing-institute/autoemulate/blob/5c743a4b34bd4197da8424419b7ec42a7fddfa6a/autoemulate/cross_validate.py#L20 https://github.com/alan-turing-institute/autoemulate/pull/180 so that it won't through an error when cv_results is not initialized (due to Exception), we will have to improve error message in general. I don't think there is a "fix" for this issue.

[kallewesterling]

Note: decided to close this issue and open a separate one about the improvement of the error message.