[BUG] RBP with multiple relevance levels

[mpetri]

Describe the bug

Given the formula of RBP of

    \operatorname{RBP} = (1 - p) \cdot \sum_{i=1}^{d}{r_i \cdot p^{i - 1}}

where r_i is the reward/utility, RBP should support multiple relevance levels similar to DCG such that if max relevance level is 2 max rbp value should be 2

[AmenRa]

Hi, thanks for pointing it out.

I suspect I assumed it was a binary relevance metric because the example in the original paper does not account for graded relevance.

I will fix it in the coming days.

[mpetri]

Looking at other implementations of RBP I also saw an option to normalize back to [0,1] by dividing by the max relevance level in the qrls.

So if max relevance level RBP in the limit goes to 2. So we divide by 2 to make it go back to [0,1]. Not sure if this is useful to implement or not.

[AmenRa]

Fixed in v0.3.15.

RBP now uses graded relevance scores from qrels instead considering them binary. Do you think I should leave an option for the old behaviour?