FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs
MIT License
7.14k stars 521 forks source link

Calculate AUC error #1049

Open dg618k opened 2 months ago

dg618k commented 2 months ago

The pred_hard_encodings size dismatch with preds_scores when faiss indice has -1. so I retrieve preds_scores first len(pred_hard_encoding) to make it correct. Does it introduce any other problems? Thank you.

staoxiao commented 2 months ago

@dg618k , Can you show the code you revised?

dg618k commented 2 months ago

@staoxiao Ok, in eval_msmarco.py around line 200. The codes I modified are:

n_score = []
for pred, label, cur_score in zip(preds, labels, preds_scores):
   ......
   n_score.append(cur_score[:len(pred_hard_encoding)])
preds_scores1d = np.asarray(n_score).flatten()