Using pytrec_eval.RelevanceEvaluator, do we sort the results internally or use as it is provides?

[SDcodehub]

I am using

        evaluator = pytrec_eval.RelevanceEvaluator(qrels, {map_string, ndcg_string, recall_string, precision_string})
        scores = evaluator.evaluate(results)

Now result is {qid: {docid: embeddding_score, docid: embeddding_score} }

so we sort the results using scores before using for metrics calculation? I am assuming it has to be as dict will not have fix indexing.

can we avoid this sorting? in that case do I have to manipulate the scores manually to achieve desired results?

[seanmacavaney]

trec_eval (and by extension pytrec_eval) works by sorting the docids by the scores you provide (descending). If you want a specific order, you can manipulate the scores in a way that gives the desired effect.

Do you think it would be helpful to allow providing a list of docids instead of a dict, and using that order?