Open EEsquirtle opened 3 months ago
Hi,
Here is the implementation of the missing functions, and the missing file add_reference_answer_test.json :
def wups(words1, words2, alpha):
"""
:param pred:
:param truth:
:param alpha:
:return:
"""
sim = 1.0
flag = False
for w1 in words1:
max_sim = 0
for w2 in words2:
word_sim = wup(w1, w2, alpha)
if word_sim > max_sim:
max_sim = word_sim
if max_sim == 0: continue
sim *= max_sim
flag = True
if not flag:
sim = 0.0
return sim
def get_wups(pred, truth, alpha):
"""
calculate the wups score
:param pred:
:param truth:
:return:
"""
pred = word_tokenize(pred)
truth = word_tokenize(truth)
item1 = wups(pred, truth, alpha)
item2 = wups(truth, pred, alpha)
value = min(item1, item2)
return value
您好,非常感谢您的开源代码!我正在尝试复现您在NExT-QA上的结果,在nextqa_data.py ->evaluate_raw中遇到一些问题,首先是add_reference_answer_test.json文件的缺失,还有其中用到的get_wups函数(from metrix import get_wups)我不确定是环境的问题还是metrix.py文件的缺失,您能提供相关的缺失文件吗,再次感谢您的杰出工作!