Closed campuslifeceo closed 1 year ago
`class CovidRetrieval(AbsTaskRetrieval): @property def description(self): return { 'name': 'CovidRetrieval', 'hf_hub_name': 'C-MTEB/CovidRetrieval', 'reference': 'https://aclanthology.org/2022.emnlp-main.357.pdf', 'description': 'COVID-19 news articles', 'type': 'Retrieval', 'category': 's2p', 'eval_splits': ['dev'], 'eval_langs': ['zh'], 'main_score': 'ndcg_at_10', }
def load_data(self, **kwargs): if self.data_loaded: return self.corpus, self.queries, self.relevant_docs = load_retrieval_data(self.description['hf_hub_name'], self.description['eval_splits']) self.data_loaded = True`
这里的reference地址应该错了吧?
CmedqaRetrieval : 这个数据集应该也有问题
您好,这两个数据集参考的是这篇论文里的Out-of-domain Evaluation章节。
`class CovidRetrieval(AbsTaskRetrieval): @property def description(self): return { 'name': 'CovidRetrieval', 'hf_hub_name': 'C-MTEB/CovidRetrieval', 'reference': 'https://aclanthology.org/2022.emnlp-main.357.pdf', 'description': 'COVID-19 news articles', 'type': 'Retrieval', 'category': 's2p', 'eval_splits': ['dev'], 'eval_langs': ['zh'], 'main_score': 'ndcg_at_10', }
这里的reference地址应该错了吧?