microsoft / mwt-ds

Umbrella repository for projects related to the MWT Decision Service
187 stars 77 forks source link

steps for offline replayability #89

Closed abgoswam closed 7 years ago

abgoswam commented 7 years ago

Code changes for offline replay-ability. The changes are built on top of the previous version of the file. Hence you will notice some code chunks have been reverted.

The core functionality this PR adds is scoring the events with the corresponding model generated offline, and using the offline predictions thus generated to compute (1) % of ranked actions that match between offline ranked actions and online ranked actions (2) IPS score from the offline predictions.