search

AmenRa / ranx

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍
https://amenra.github.io/ranx
MIT License
438 stars 24 forks source link

[Feature Request] Benchmarking #12

Closed ilivans closed 2 years ago

ilivans commented 2 years ago

Is your feature request related to a problem? Please describe. To claim "blazing-fast", it would be nice to have benchmarks against existing implementations.

Describe the solution you'd like The implementation is benchmarked against some/all of the sources below:

  1. naive numpy implementation
  2. scikit-learn metrics module
  3. AstraZeneca/rexmex
  4. microsoft/rankerEval
  5. pytrec_eval
  6. ?
AmenRa commented 2 years ago

Hi, thank you for your interest in ranx!

I compared ranx against pytrec_eval for my ECIR paper, you can re-run the benchmark using this notebook.

Here are the results reported in the ECIR paper that will be out in April:

Screenshot 2022-02-16 at 15 26 36

Adding a comprehensive comparison with other libraries is on my to-do list, but I have not had the time to do it yet.

Please, let me know what you think about the benchmark results.

Elias