What's the meaning of Non-Factual* in the paper?

[skpig]

Same as the title. Thanks in advance.

[potsawee]

Hi @skpig

Sorry for the confusion about Non-Factual, the denotes "less-trivial hallucination detection" as in our setup/definition, the percentage of hallucination is quite high (in some examples, all sentences are labelled as major-hallucination), we would like to see the performance of these methods in detecting more challenging hallucination detection. You can see in this notebook about how we created and used "Non-Factual*" labels (human_label_detect_False_h): https://github.com/potsawee/selfcheckgpt/blob/main/demo/experiments/probability-based-baselines.ipynb

Hope this clarifies your question Potsawee

[skpig]

Thanks for your response!