search

violet-zct / fairseq-detect-hallucination

Detect hallucinated tokens for conditional sequence generation.
MIT License
63 stars 5 forks source link

Can 'predict_hallucination_mt.py' be used to predict hallucination for summarizaion tasks? #3

Closed Lyshc-cgc closed 2 years ago

Lyshc-cgc commented 2 years ago

@violet-zct Hi, I'm sorry to touble u again. As mentioned in REAMDE.ME, we can use 'predict_hallucination_mt.py' directly to predict hallucination. There is a XLMRModel for predicting MT hallucination in that script.

xlmr = XLMRModel.from_pretrained(
    model_path,
    checkpoint_file='checkpoint.pt',
    data_name_or_path=datapath
)

Can we use it directly to predict hallucination for summarizaion tasks? Or, Should we write another script to use roberta model for predicting hallucination in summarization?

violet-zct commented 2 years ago

No worries. So XLM-Roberta is a cross-lingually pre-trained model that handles different languages, for English summarization tasks, please use the Roberta model.

Lyshc-cgc commented 2 years ago

Appreciate sincerely for your reply!