search

Display-Lab / precision-feedback-pipeline

Apache License 2.0
1 stars 0 forks source link

Esteemer should randomly choose social better messages with mpm score ties, rather than by alphabetical order #203

Closed icejag closed 10 months ago

icejag commented 10 months ago

Problem: When testing randomization of all boundary cases we came across that the randomization failed for all values above top 10% and top25%

To test the issue of randomization with the : [5,"GLU01","2023-12-01",98,16,100,85,88,96,90], [5,"BP03","2023-12-01",98,16,100,85,88,96,90], [5,"TOC01","2023-12-01",98,16,100,85,88,96,90]

or [5,"GLU01","2023-12-01",94,16,100,85,88,96,90], [5,"BP03","2023-12-01",94,16,100,85,88,96,90], [5,"TOC01","2023-12-01",94,16,100,85,88,96,90]

Proposed Solution: When solving this issue we choose the sorted highest value. Instead of choosing sorted highest value, we should use random algorithm in choosing.

zachll commented 10 months ago

Former title: Randomization at the top 25%,top10% is failing (it chooses alphabetical order)