search

potsawee / selfcheckgpt

SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
MIT License
442 stars 54 forks source link

Could you provide a example of showing the AUC-PR of your own methods(i.e. the selfcheckgpt) just like the probability-based-baselines? #32

Open goatyu3 opened 2 months ago

goatyu3 commented 2 months ago

Hi there! Thanks for your awesome works. Just like what i said in the title:Could you provide a example of showing the AUC-PR of your own methods(i.e. the selfcheckgpt) just like the probability-based-baselines?

potsawee commented 2 months ago

Hi @goatyu3

thanks for trying the selfcheckgpt repository! I've added a notebook to show an example of SelfCheckGPT on WikiBio like the one with probability-based baselines. You can find it in the link below.

https://github.com/potsawee/selfcheckgpt/blob/main/demo/experiments/selfcheckgpt-nli-on-wikibio.ipynb

goatyu3 commented 2 months ago

@potsawee Thank you for your reply. I will have a look.