ptrckhmmr / learning-to-defer-with-limited-expert-predictions

Code for "Learning to Defer with Limited Expert Predictions" (AAAI 2023)

8 stars 3 forks source link

human-ai-teaming learning-to-defer semi-supervised-learning

readme

Learning to Defer with Limited Expert Predictions

This repository includes the code to reproduce the experiments for our paper (paper-source).

Overview of the proposed framework

Contents

This repository is structured as follows:

Embedding-Semi-Supervised: Code for the proposed Emb-SSL implementations Embedding-FixMatch and Embedding-CoMatch
Embedding-Supervised: Code for the Emb-SL baselines Embedding-SVM and Embedding-NN
Semi-Supervised: Code for the SSL baselines FixMatch and CoMatch
Learning-to-Defer-Algs: Implementations of the learning to defer algorithms of Mozannar & Sontag, Okati et al., and Raghu et al..

Requirements

This code depends on the following packages:

torch
torchvision
torchtext
timm
scipy
seaborn
numpy
matplotlib
scikit-learn
tensorboard_logger

Execution

Follow these steps to reproduce our experiments:

(NIH experiments only) Download and extract the NIH dataset to nih_images/
(CIFAR experiments only) Generate the synthetic expert labels
Execute the training of one of the proposed Embedding-Semi-Supervised approaches (or any baseline)
Generate the artificial expert labels
Execute the training of one of the learning to defer algorithms

Detailed instructions can be found in the respective subfolders.