'ConformalMultiQuantile' object has no attribute 'calibration_adjustments'

[akshat-suwalka-dream11]

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. Go to "..."
2. Click on "..."
3. Scroll down to "..."
4. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• MAPIE Version [e.g. 0.3.2]

Additional context Add any other context about the problem here.

I have the save the model but while loading and predicting it is throwing error 'ConformalMultiQuantile' object has no attribute 'calibration_adjustments'

class ConformalMultiQuantile(CatBoostRegressor):

def __init__(self, quantiles:Iterable[float], *args, **kwargs):

    """
    Initialize a ConformalMultiQuantile object.

    Parameters
    ----------
    quantiles : Iterable[float]
        The list of quantiles to use in multi-quantile regression.
    *args
        Variable length argument list.
    **kwargs
        Arbitrary keyword arguments.
    """

    kwargs['loss_function'] = self.create_loss_function_str(quantiles)
    super().__init__(*args, **kwargs)
    self.quantiles = quantiles
    self.calibration_adjustments = None

@staticmethod
def create_loss_function_str(quantiles:Iterable[float]):

    """
    Format the quantiles as a string for Catboost

    Paramters
    ---------
    quantiles : Union[float, List[float]]
        A float or list of float quantiles

    Returns
    -------
    The loss function definition for multi-quantile regression
    """

    quantile_str = str(quantiles).replace('[','').replace(']','')

    return f'MultiQuantile:alpha={quantile_str}'

def calibrate(self, X_calib, y_calib):

    """
    Calibrate the multi-quantile model

    Paramters
    ---------
    X_calib : ndarray
        Calibration inputs
    y_calib : ndarray
        Calibration target
    """

    # Ensure the model is fitted
    if not self.is_fitted():

        raise CatBoostError('There is no trained model to use calibrate(). Use fit() to train model. Then use this method.')

    # Make predictions on the calibration set
    uncalibrated_preds = self.predict(X_calib)

    # Compute the difference between the uncalibrated predicted quantiles and the target
    conformity_scores = uncalibrated_preds - np.array(y_calib).reshape(-1, 1)

    # Store the 1-q quantile of the conformity scores
    self.calibration_adjustments = \
        np.array([np.quantile(conformity_scores[:,i], 1-q) for i,q in enumerate(self.quantiles)])

def predict(self, data, prediction_type=None, ntree_start=0, ntree_end=0, thread_count=-1, verbose=None, task_type="CPU"):

    """
    Predict using the trained model.

    Parameters
    ----------
    data : pandas.DataFrame or numpy.ndarray
        Data to make predictions on
    prediction_type : str, optional
        Type of prediction result, by default None
    ntree_start : int, optional
        Number of trees to start prediction from, by default 0
    ntree_end : int, optional
        Number of trees to end prediction at, by default 0
    thread_count : int, optional
        Number of parallel threads to use, by default -1
    verbose : bool or int, optional
        Verbosity, by default None
    task_type : str, optional
        Type of task, by default "CPU"

    Returns
    -------
    numpy.ndarray
        The predicted values for the input data.
    """

    preds = super().predict(data, prediction_type, ntree_start, ntree_end, thread_count, verbose, task_type)

    # Adjust the predicted quantiles according to the quantiles of the
    # conformity scores
    if self.calibration_adjustments is not None:

        preds = preds - self.calibration_adjustments

    return preds

Store quantiles 0.005 through 0.99 in a list

quantiles = [q/200 for q in range(1, 200)]

Instantiate the conformal multi-quantile model

conformal_model = ConformalMultiQuantile(iterations=100, quantiles=quantiles, verbose=10)

Fit the conformal multi-quantile model

conformal_model.fit(X_train, y_train)

import os import pickle

Define the directory path and file name

directory_path = "/da/" file_name = "model_a.pkl" file_path = os.path.join(directory_path, file_name)

Create the directory if it doesn't exist

os.makedirs(directory_path, exist_ok=True)

Now you can save the model

with open(file_path, "wb") as f: pickle.dump(conformal_model, f)

import os import pickle

Define the directory path and file name

directory_path = "/da/" file_name = "model_a.pkl" file_path = os.path.join(directory_path, file_name)

Load the model

with open(file_path, "rb") as f: loaded_model = pickle.load(f)

preds_uncalibrated_1 = conformal_model.predict(X_test) preds_uncalibrated_2 = loaded_model.predict(X_test)

Error 12 preds_uncalibrated_1 = conformal_model.predict(X_test) preds_uncalibrated_2 = loaded_model.predict(X_test) AttributeError: 'ConformalMultiQuantile' object has no attribute 'calibration_adjustments'

AttributeError Traceback (most recent call last) File :2 1 preds_uncalibrated_1 = conformal_model.predict(X_test) ----> 2 preds_uncalibrated_2 = loaded_model.predict(X_test)

File :104, in ConformalMultiQuantile.predict(self, data, prediction_type, ntree_start, ntree_end, thread_count, verbose, task_type) 100 preds = super().predict(data, prediction_type, ntree_start, ntree_end, thread_count, verbose, task_type) 102 # Adjust the predicted quantiles according to the quantiles of the 103 # conformity scores --> 104 if self.calibration_adjustments is not None: 106 preds = preds - self.calibration_adjustments 108 return preds

[LacombeLouis]

Hello @akshat-suwalka-dream11, Thank you for the comment and adding the code. It seems like this is not a MAPIE specific issue? I can't seem to understand where in your code you make use of the MAPIE library. Could you please provide more details?

It would seem your issue is regarding CatBoostRegressor and in this case I suggest you ask your question to their Github.

I hope you find a solution.