[HoN Game] Changing the Outcome Logic

go-bazzinga / ds-dags

Other

0 stars 0 forks source link

[HoN Game] Changing the Outcome Logic #2

Open jay-dhanwant opened 1 month ago

jay-dhanwant commented 1 month ago

Moving away from majority driven outcome logic to one that incorporates signals associated with virality of the video such as likes, views, watch time, etc.

Will prevent users from gaming the system and will increase NOTs chosen
Will induce unpredictability in the game increasing gameplays and retention
Help increase content consumption

Creating batch jobs for relevant metrics (2 Days )

[x] #3
[x] #4
[ ] #5

Simulation (2 Days)

[x] #6
[ ] #7
[ ] #8

Service to return HON for each video

[ ] #9

jay-dhanwant commented 1 month ago

Creating batch jobs for relevant metrics (2 Days )

[ ] Batch job for HON metric calculation
[x] Brainstorm on the sampling service to get a bag of scores (recency vs stability)
[ ] Create and maintain sampling for each videos

Simulation (2 Days)

[ ] Sample old videos and gather interaction data for them
[ ] Simulate the algorithm and create batches of wins and loses for different hyper-parameters
[ ] Create an analysis report for simulated games

Service to return HON for each video

[ ] Could be a server listening for video ids / tasks to be added after finalising the implementation

Natasha-GB commented 1 month ago

v1: Compare with similar video and its hyper-parameters v2: Random injection of videos into feed to get realistic data around the video

harshitasrivastav28 commented 1 month ago

Started working on it yesterday

harshitasrivastav28 commented 3 weeks ago

Simulation by this week will be done

Natasha-GB commented 3 weeks ago

Setting up a threshold to decide HOT or NOT (to be discussed today)

harshitasrivastav28 commented 2 weeks ago

Working on simulation to be closed by EOD
Report button instrumentation is missing

harshitasrivastav28 commented 2 weeks ago

There were handful amount of metrics which were skewing up.
We have further analysed the outliers
Simulation of HON game on new logic to see what all videos should have end up winning and what videos would have end up losing

Natasha-GB commented 1 week ago

Need to figure out deployment
Jay to share video ids working as per new logic for validation
Approximating global comparison for now, will be better as we have more data
300 to 700 based on deviation is the number of videos we'll need for moving to the new logic

harshitasrivastav28 commented 1 week ago

How to schedule and integrate the new logic and check the response needs discussion