Open faneshion opened 4 months ago
The task
will be removed outside of the Metric. We can divide all metrics into four categories: AnswerCorrectness
, AnswerGroundedness
, ContextRelevancy
, and ContextAdequacy
.
For each metric, we can decorate it to add the metric_type attribute:
def add_attribute(attribute_name, attribute_value):
def decorator(cls):
setattr(cls, attribute_name, attribute_value)
return cls
return decorator
# 使用装饰器定义类的属性
@add_attribute('mtype', 'AnswerGroundedness')
class _em_answer(Metric):
pass
# 创建类的实例并访问属性
metric = rl.metrics._em_answer()
print(metric.mtype) # 输出: AnswerGroundedness
Enhance the base metric for robust evaluation:
self.task
to distinguish input format.def _validate_data(self, input: Dataset) -> bool
to check the validity of the data.