search

intuit / metriks

Python package of commonly used metrics for evaluating information retrieval models.
MIT License
25 stars 25 forks source link

NDCG and Confusion Matrix Examples #8

Open wontonswaggie opened 3 years ago

wontonswaggie commented 3 years ago

Create a usage example on how these ranking metrics can be used:

+------------------------------------------------------------+-------------------------------------------------------------------------------+
| Python API                                                 | Description                                                                   |
+============================================================+===============================================================================+
| `metriks.ndcg(y_true, y_prob, k)`                          | A score for measuring the quality of a set of ranked results.                 |
+------------------------------------------------------------+-------------------------------------------------------------------------------+
| `metriks.confusion_matrix_at_k(y_true, y_prob, k)`         | Generates binary predictions from probabilities by evaluating the top k       |
|                                                            | items (in ranked order by y_prob) as true.                                    |
+------------------------------------------------------------+-------------------------------------------------------------------------------+
  1. Identify a dataset that can be used to train a ranking model
  2. Train a ranking model with the data
  3. Use the given metrics above and show results and demonstrate how these metrics can be used
asaljalilvand commented 3 years ago

Hi, I'm an attendee at the GHC 2020 and I'm interested in working on this issue.

kshithi03 commented 3 years ago

I'm a GHC 2020 attendee and would like to contribute to this issue. Thanks!

swati-murmu commented 3 years ago

I'm a GHC 2020 attendee and would like to contribute to this issue. Thanks!

Hi. I am a vGHC 2020 attendee too. I have created a slack channel named #opentrack-metriks-8. If you could join that channel, we could start working on it together!