deel-ai / puncc

👋 Puncc is a python library for predictive uncertainty quantification using conformal prediction.
https://deel-ai.github.io/puncc/
280 stars 16 forks source link

[Feature Request]: - classification #56

Open valeman opened 3 weeks ago

valeman commented 3 weeks ago

Module

Classification

Contact Details

No response

Feature Request

The models offered by PUNCC for classification seem to be mostly from computer vision rather than models for tabular data. is there are a reason for this?

Most of industry use cases (80-90% use cases) are about tabular data and classification problem is a significant part of it.

The models offered by PUNCC were not designed for and are not suitable for tabular data (e.g. RAPS etc). Such models produce prediction sets where individual objects probabilities don’t have calibration and as a result are not useful for decision making as businesses need individually calibrated class probabilities for optimal decision making.

It is recommended for PUNCC to review classification models, identity the gaps with the view of offering users suitable functionalities for tabular data classification rather than models that seem to have been randomly plucked from computer vision.

A minimal example

No response

Version

v0.9

Environment

- OS:
- Python version:
- Packages used version:
M-Mouhcine commented 3 days ago

Hi @valeman,

Many thanks for your detailed feedback and suggestions.

I agree that tabular data is a significant use case for classification problems in industry. We will work on including classification models and suitable conformal methods for tabular data, and consider it a priority on PUNCC's roadmap.