namoray / nineteen

nineteen
7 stars 16 forks source link

Optimized Selection of Miners for Queries - Carlo Casorzo #71

Closed deepbludev closed 3 days ago

deepbludev commented 1 week ago

Resolves #70

Solution: Improved Selection of Miners for Queries Using Time-Decayed Period Scores

Overview

This solution enhances the selection process of miners for both organic and synthetic queries by integrating time-decayed period scores. The goal is to ensure high-quality responses for users while promoting exploration for new miners, all while accurately reflecting miners' recent historical performance.

Key Improvements

  1. Time-Decayed Historical Period Scores

    • Organic Queries: Prioritize miners based on their time-decayed historical period scores, emphasizing recent performance.
    • Separate Decay Factors: Utilize separate decay factors for discrete periods (used in score calculations) and continuous time intervals (used in contender selection), ensuring appropriate weighting over time.
  2. Differentiated Selection Criteria

    • Organic Queries: Miners with higher time-decayed historical period scores are prioritized to deliver top-quality responses to users.
    • Synthetic Queries: Miners with fewer total requests made are prioritized to assess and evaluate less-experienced or new miners.
  3. Promoting Exploration of New Miners

    • By filling the unfulfilled slots with miners with fewer requests made, we promote the exploration of new miners, giving them the chance to also build their scores with organic queries.

Benefits

PD: "seasoning" :)

deepbludev commented 1 week ago

So this solution is ineffective for a few reasons:

  • You're using the PERIOD_SCORE to assess which miners are best. You're using this from the contenders table. The contenders table only has information from the current cycle. Actually from the current cycle all PERIOD_SCORES are null. So this is no change to the current solution.
  • Are you sure period score is a good measure of how good the responses are?
  • Lots of formatting changes makes it hard to review

I just pushed a new approach that takes into account the historical scores instead.