FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs
MIT License
6.97k stars 508 forks source link

数据引用错误 #67

Closed campuslifeceo closed 1 year ago

campuslifeceo commented 1 year ago

`class CovidRetrieval(AbsTaskRetrieval): @property def description(self): return { 'name': 'CovidRetrieval', 'hf_hub_name': 'C-MTEB/CovidRetrieval', 'reference': 'https://aclanthology.org/2022.emnlp-main.357.pdf', 'description': 'COVID-19 news articles', 'type': 'Retrieval', 'category': 's2p', 'eval_splits': ['dev'], 'eval_langs': ['zh'], 'main_score': 'ndcg_at_10', }

def load_data(self, **kwargs):
    if self.data_loaded:
        return

    self.corpus, self.queries, self.relevant_docs = load_retrieval_data(self.description['hf_hub_name'],
                                                                        self.description['eval_splits'])
    self.data_loaded = True`

这里的reference地址应该错了吧?

campuslifeceo commented 1 year ago

CmedqaRetrieval : 这个数据集应该也有问题

staoxiao commented 1 year ago

您好,这两个数据集参考的是这篇论文里的Out-of-domain Evaluation章节。