donlnz / nonconformist

Python implementation of the conformal prediction framework.
MIT License
436 stars 96 forks source link

MyClassifierAdapter - Using a pickled model #37

Closed yzhangi96 closed 2 years ago

yzhangi96 commented 2 years ago

Hi, thank you so much for the great repo! I was trying to create a new adapter class for classification but instead of doing the fitting in the class, I want to load in a pickled model. This was my attempt:

class MyClassifierAdapter(ClassifierAdapter):
    def __init__(self, model, model_path, fit_params=None):
        super(MyClassifierAdapter, self).__init__(model, fit_params)
        self.model_path: str = model_path

    def fit(self, x, y):
        '''
            x is a numpy.array of shape (n_train, n_features)
            y is a numpy.array of shape (n_train)

            Here, do what is necessary to train the underlying model
            using the supplied training data
        '''
        with open(self.model_path, "rb") as p:
            self.model.fit = pickle.load(p)

    def predict(self, x):
        '''
            Obtain predictions from the underlying model

            Make sure this function returns an output that is compatible with
            the nonconformity function used. For default nonconformity functions,
            output from this function should be class probability estimates in
            a numpy.array of shape (n_test, n_classes)
        '''
        return self.model.predict_proba(x)

However, running the following gave me the error: TypeError: predict_proba() missing 1 required positional argument: 'X'

    my_classifier = XGBClassifier
    model = MyClassifierAdapter(
        model=my_classifier,
        model_path=MYPATH
    )
    nc = ClassifierNc(model)
    icp = IcpClassifier(nc)

    # This should read in the specified model
    icp.fit(data_trainx, data_trainy)

    # Calibrate the ICP using the calibration set
    icp.calibrate(data_calibratex, data_calibratey)

    # Produce predictions for the test set, with confidence 95%
    prediction = icp.predict(data_testx, significance=0.05)

Full stacktrace of the error:

Traceback (most recent call last):
  File "my_classifier_adapter.py", line 62, in <module>
    icp.calibrate(data_calibratex, data_calibratey)
  File "/lib/python3.6/site-packages/nonconformist/icp.py", line 104, in calibrate
    cal_scores = self.nc_function.score(self.cal_x, self.cal_y)
  File "/lib/python3.6/site-packages/nonconformist/nc.py", line 365, in score
    prediction = self.model.predict(x)
  File "my_classifier_adapter.py", line 41, in predict
    return self.model.predict_proba(x)
TypeError: predict_proba() missing 1 required positional argument: 'X'