This is a library for generating prediction sets for machine learning regression tasks. We do this by first converting regression to a classification problem (divide the output space into 50 bins) and then using CP techniques for classification to obtain a conformal set.
You can install by using pip.
pip install R2CCP
Our example file (example.py) provides a simple demonstration of how to use our R2CCP class for conformal prediction. At a high level, the basic steps are instantiating the model class, fitting against data, and analyzing the results.
# Import the model
from R2CCP.main import R2CCP
# Instiantiate the model
model = R2CCP({'model_path':'model_paths/model_save_destination.pth', 'max_epochs':5})
// model_path is where to save the trained model output (required parameter)
# Fit against the data
model.fit(X_train, y_train)
# Analyze the results
intervals = model.get_intervals(X_test)
coverage, length = model.get_coverage_length(X_test, Y_test)
print(f"Coverage: {np.mean(coverage)}, Length: {np.mean(length)}")
# If you don't have labels, you can just use get_length
length = model.get_length(X_test)
# Get model predictions
predictions = model.predict(X_test)
# You can also change the desired coverage level
model.set_coverage_level(.8)
new_coverage, new_length = model.get_coverage_length(X_test, Y_test)
print(f"New Coverage: {np.mean(coverage)}, New Length: {np.mean(length)}")
Here, we give a small example on a regression problem. We first generate a synthetic dataset of features of labels. We then generate the conformal intervals from this dataset.
from R2CCP.main import R2CCP
import numpy as np
X_train = np.random.rand(10, 1)
Y_train = 2 * X_train + 1 + 0.1 * np.random.randn(10, 1)
X_test = np.random.rand(10, 1)
Y_test = 2 * X_test + 1 + 0.1 * np.random.randn(10, 1)
model = R2CCP({'model_path':'model_paths/model_save_destination.pth', 'max_epochs':5})
model.fit(X_train, Y_train)
intervals = model.get_intervals(X_test)
coverage, length = model.get_coverage_length(X_test, Y_test)
print(f"Coverage: {np.mean(coverage)}, Length: {np.mean(length)}")
The R2CCP class can be instantiated with a variety of different parameters. Here is an overview of all the available options.