ahans30 / Binoculars

[ICML 2024] Binoculars: Zero-Shot Detection of LLM-Generated Text
https://arxiv.org/abs/2401.12070
BSD 3-Clause "New" or "Revised" License
189 stars 26 forks source link

Threshold tuning #3

Closed zaxtax closed 1 month ago

zaxtax commented 7 months ago

Would you be able to put up the code you used to tune the threshold for the classifier? The value is highly sensitive to the choice of performer and observer model

ahans30 commented 1 month ago

You can use following to get set of thresholds for different false positive rates and true positive rates. Hope this helps.

fpr, tpr, thresholds = metrics.roc_curve(y_true=score_df["class"], y_score=score, pos_label=1)

Further, you can interpolate threshold for desired pair of tpr/fpr.