Question on rank aggregation usage

[Wwwwei]

Thanks for your amazing work. I am very interested in this framework and try to use it to solve my rank aggregation problems. However, I am a little confused about the usage.

For example, I have scores for several items under different ranking rules, as follows:

item | rank1 | rank2 | rank3
item1 | 0.4 | 0.8 | 0.2
item2 | 0.8 | 0.7 | 0.7
item3 | 0.7 | 0.7 | 1.0
item4 | 0.5 | 0.6 | 0.7

Now I want a comprehensive rank that satisfies all the subrank (e.g., rank1, rank2, rank3) constraints as much as possible.

item | rank
item1 | 0.5
item2 | 0.9
item3 | 0.7
item4 | 0.4

I'm not sure if that can be addressed with ranx. If so, could you show me an example? Thanks a lot.

[AmenRa]

Hi, you can find an example on Colab and other information here.

As for your example, you could to something like this:

from ranx import Run, fuse

run_1 = Run({"q1": {"item1": 0.4, "item2": 0.8, "item3": 0.7, "item4": 0.5}})
run_2 = Run({"q1": {"item1": 0.8, "item2": 0.7, "item3": 0.7, "item4": 0.6}})
run_3 = Run({"q1": {"item1": 0.2, "item2": 0.7, "item3": 1.0, "item4": 0.7}})

combined_run = fuse(runs=[run_1, run_2, run_3], method="max")

print(combined_run.to_dict()["q1"])

Output:

{
    "item2": 1.0,
    "item3": 1.0,
    "item1": 1.0,
    "item4": 0.625,
}

EDIT: copy-pasted wrong output dict.

[Wwwwei]

Thank you for your reply! it is very helpful and I will try it soon. By the way, any suggestions for speeding up when the number of items and the subranks (e.g., tens of thousands of items and dozens of subranks) are both large?

[AmenRa]

Neither the implemented methods nor the implementation itself are meant to be used in the context you describe. Usually, we aggregate two or three rankings for 1k documents each (the top 1k in each ranking). Give ranx a try for your use case. Maybe use it to find the method that works best with a sample of your data and then build your custom implementation that can run on the GPU. I do not think you can do much better than my implementation performance-wise, at least not in Python.

[Wwwwei]

Thanks a lot. I got it.