search

SlangLab-NU / links

1 stars 0 forks source link

Find literatures that run OOD detection experiments using CLINC150 dataset #15

Open ztybigcat opened 1 year ago

ztybigcat commented 1 year ago

In the Amazon paper "Supervised Clustering Loss for Clustering-Friendly Sentence Embeddings: an Application to Intent Clustering", they calculate PRAUC of the model rather differently: "using the true labels of pairs sharing the same cluster as 1 and pairs with different clusters as 0." This results in PRAUC data that is quite different from the results we have.

Week Oct 23: Find other literature that run ood experiments using CLINC150, Massive, BANKING77, DSTC11, HWU64 dataset so we verify the baseline results we have.

ztybigcat commented 1 year ago

Week Nov 6: Run fine-tuning on roberta model and recalculate maha distance. ID and OOD shows significant separation.